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ABSTRACT 


This thesis introduces Concept Engineering, the result of an extensive 
collaborative research effort with product development professionals from 
member companies of the Center for Quality Management, as a complete 
decision support process for enhancing product concept development. Concept 
Engineering applies the principles and practices of Total Quality Management to 
develop customer-focused product concepts. The simultaneous introduction of 
Concept Engineering into product development organizations in three different 
companies created an opportunity for a comparative study of the product 
concept decision process. The comparative analysis is conducted using Inductive 
System Diagrams, a method introduced in this research, for systematic field- 
based hypothesis generation. Inductive System Diagrams combine aspects of 
grounded theory methods and system dynamics to develop and communicate 
substantive theories intimately tied to the data. The cross-company comparative 
analysis of product development teams, some using and others not using 
Concept Engineering, led to the generation of a dynamic hypotheses regarding 
the impact of a relative emphasis on TIME vs. MARKET orientation during the 
product concept decision process. It is proposed that a relative emphasis on 
TIME reduces concept development time but increases total product 
development time compared to a relative emphasis on MARKET orientation 
during product concept development. The MARKET orientation results in 
design objectives with higher clarity, credibility and stability. TIME orientation 
led to relatively lower design objective clarity and credibility resulting in product 
concept changes during downstream development activities thus increasing the 
total development time. 
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Preface 

The presentation of this thesis follows the progression of work done in this 
research effort. In the beginning, there was a desire to investigate the "customer 
focus" theme of Total Quality Management and it was realized that product 
development represented a high leverage point in which to conduct this work. A 
collaborative research effort with product development professionals from 
member companies of the Center for Quality Management led to the evolution of 
Concept Engineering as a complete decision support process for product concept 
development. This material is covered in Chapter 1 and Appendix A . 

The introduction of Concept Engineering to product development efforts 
within the participating companies created an opportunity for a inter-company 
comparative study. Additionally, as it was not feasible for the participating 
companies to implement a whole-sale conversion to Concept Engineering, it was 
possible to conduct an intra-company comparative analysis involving product 
development teams studying Concept Engineering and those that did not. The 
design of this research is addressed in Chapter 2. 

The research design led to an opportunity to conduct extensive, field- 
based participant observation of product development activities, in a 
comparative setting. As a result, it was possible to apply relevant theory 
generation methods from sociology to the product concept decision process. It 
was observed that a relative strength of the sociological methods was in the 
identification of key process variables. However, relative to System Dynamics 
methodologies for variable integration, the Sociological integration methods 
were weak. Asa result, a marriage of the relative strengths of the two 
approaches was created in a process called Inductive System Diagrams. This 


process is described in Chapter 3. 
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In chapter 4, the hypotheses related to product concept development, 
developed through the Inductive System Diagram process, are presented in the 
context of current literature. Based on the generated hypotheses, management 
diagnostics for the product concept decision process are presented in Chapter 5. 
Chapter 6 describes potential next steps to continue the development of Concept 


Engineering, product concept decision theory and Inductive System Diagrams. 
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Chapter 1: Concept Engineering 


The total quality management (TQM) literature has two dominant 
themes: continuous process improvement and customer focus. The first 
theme, continuous process improvement, has a well-established set of 
methods and tools (7 Steps, 7 Statistical Tools, etc.) that are widely 
disseminated in practice and academia. (Feigenbaum 1951, Western Electric 
Co. 1956, Juran & Gryna 1970, Ishikawa 1976, Kume 1985, AT&T 1987, Scholtes 
1988, Montgomery 1991) In contrast, the customer focus theme, with the 
principal exception of Quality Function Deployment activities (Hauser & 
Clausing 1988, Akao 1990, Griffin & Hauser 1992) is not supported by a similar 
set of widely accepted tools and methods. However, an emphasis on 
attending to customer needs as a critical success factor in new product 
development has been consistently underscored by researchers in the quality 
literature (Shewhart 1938, Deming 1982, Ishikawa 1985, Juran 1988) and in the 
product development literature (for example: Rothwell et al. 1974, Cooper & 


Klienschmidt 1986, Clark & Fujimoto 1991). 


Motivation 

In Cooper and Klienschmidt's (1986; p.76) study of 252 new product case 
histories in 123 firms "the weakest rated activities were the ‘up front' or pre- 
development activities, namely initial screening, preliminary market 
assessment and detailed market study." Supporting this finding, many 
studies conclude there is potential benefit from research on the product 
development process, particularly the early activities (Rothwell, et al.. 1974, 


Cooper & Klienschmidt 1986, Clausing & Pugh 1991, NRC 91, Mahajan & 
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Wind 1992). Additionally, a recent National Research Council report: 
Improving Engineering Design: Designing for Competitive Advantage. 
estimates that 70% or more of product life cycle costs are determined during 


concept design, as illustrated in the figure below (NRC 1991; p.5). 


Life Cycle Cost Commitment 
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The NRC report concludes the overall quality of engineering design in 
the United States is poor. Empirical studies of actual product development 
efforts confirm that critical activities are consistently omitted (Cooper & 


Klienschmidt 1986, Mahajan & Wind 1992). Additionally, the results of my 
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own field investigations in this area are consistent with this conclusion; I 
have heard CEOs of successful companies describe their product development 
process as "random walks with random results" and as a series of "blind 
alleys” (Burchill et al. 1992). In pursuit of this identified need, Concept 
Engineering was developed as a process for integrating customer driven 


requirements into design activities. 


Concept Engineering 

Concept Engineering is at one level a process for developing 
product/service concepts that strive to meet or exceed customer 
requirements; at another level it is a decision support process.! This chapter 
outlines the evolution and essential features of the Concept Engineering 
process (a complete description is included as Appendix A) and then provides 
evidence that it is consistent with the requirements of complete decision 
support processes. A complete decision support process is defined as one that 


supports the decision maker in all phases of the problem solving process. 


Concept Engineering Evolution 

Concept Engineering had its genesis in the teachings of Dr. Shoji Shiba, 
a visiting professor at MIT, in the fall of 1990. Professor Shiba presented 
several Total Quality Management decision aides in the context of a quality 
deployment case study. Coupling Shiba’s work with Dr. Deming's concept of 


operational definitions (Deming 1982) led to the outline of a process for 


' Although the term Decision Support System has generally been applied to problem-solving 
assistance systems using computers (Elam, et al.. 1986), there is evidence that pencil and paper 
delivery systems are just as effective as computerized versions (Cat-Baril & Huber 1987). 
Therefore, I will use the term Decision Support Process (DSP) to refer to a problem-solving 
system without the requirement to include computers. 


IS 


operationally defining customer requirements which the author applied in 
the development of salt-water flyfishing stripping basket.@ 

A review of the Stripping Basket project by a member of the Operating 
Committee of the Center for Quality Management? (CQM) led to an offer to 
present this material in the CQM's Six-day TQM Course for Senior Executives 
in the summer of 1991. This offer blossomed into a two year collaborative 
effort by representatives of several CQM member companies and MIT to 
apply the Plan-Do-Check-Act cycle (Ishikawa 1985) to the development of 


what eventually became Concept Engineering. 


Mutual Learning 

In the development of Concept Engineering, the company participants 
were equal partners with the researcher in identifying and evaluating the 
investigated problems and solutions. Representatives from four companies 
and MIT were engaged in a continuous relationship over a two year period. 
The members met collectively to discuss objectives and findings and worked 
independently pursuing particular assignments. In stretches, often lasting 
several months, the group met for as much as one full day per week. Interim 
periods were spent implementing and evaluating the results of previous 
decisions. 

During the evaluation periods, it was not unusual for members of one 
company to be present in the product development team meetings of other 
participating companies observing, along side the MIT researcher, the effects 


of proposed solutions. This level of sharing allowed insights into what 


2 The stripping basket, which has been patented and licensed, has been reviewed in the New 
York Times and was widely acclaimed in the flyfishing trade press in 1992 and 1993. 

3 The Center for Quality Management, headquartered in Cambridge MA., is a consortium of over 
thirty organizations dedicated to the pursuit of Total Quality Management. 
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worked and didn’t work to be rapidly spread among participating companies, 
l.e., an innovation at one company could be applied at another company 
usually in the matter of days or weeks at most. (The resulting rapid feedback 
on process improvement opportunities was a major contributing factor in 
Concept Engineering’s development into a complete decision support 
process.) The practice of mutual learning and sharing continues as evidenced 
by two presentations at the February, 1993 Concept Engineering User's Group 
in which two companies presented innovations and enhancements to the 
Concept Engineering process (Appendix B). 

A significant advantage of practitioner research partners is the ability to 
focus effort on substantive issues. In this research effort, the problems 
investigated were those which product development professionals in the 
firms were facing. "Practitioners often bring the pursuit of irrelevant or ill- 
conceived lines of inquiry to a rapid halt, correcting or refining the questions 
asked in ways that lead to sharper formulation and more productive 
research” (Whyte et al. 1991; p. 54). Additionally, the investment made by 
the organizations in researching existing problems provides a built-in 
incentive for implementing the solutions. This model is in sharp contrast to 
the conventional approach of literature reviews, hypotheses development 
and subsequent search for an organizational setting for testing (Whyte et al. 
1991). 

Elden and Levin (1991) describe this process of participative action 
research as "cogenerative learning”. In cogenerative learning, insiders and 
outsiders bring their respective frameworks (understandings) of events 
together to create a shared explanatory framework more powerful than any 
they could have generated independently. Insiders experience the workplace 


directly and have a great deal of specific knowledge of the setting; this 


As 


knowledge is often tacit and not reflected on. Outsiders (researchers) have 
general knowledge of the field of interest and training in systematic inquiry 
and analysis techniques. This marriage of specific and general knowledge 


provides an opportunity for the creation of new substantive knowledge. 


Concept Engineering Description 

Concept Engineering is a conceptual model, with supporting decision 
aids, for developing product concepts. The process alternates between the 
level of thought (reflection) and level of experience (data) (Kawakita 1991) in 
a way that allows participants to understand what is important to the 
customer, why it is important, how it will be measured and how it will be 
addressed in the product concept. Concept Engineering has five stages each 
with three steps (see figure on the next page). These stages and steps form the 
road map which outlines the conceptual model underlying our product 
concept decision process. 

The model begins with developing a plan for the entire concept 
development process and ends with the selection of the product concept to be 
pursued in subsequent development activities. Within each step, decision 
aids are provided to assist decision making. In some instances, alternative 
decision aids were employed and evaluated for apparent effectiveness in 
providing assistance in the product concept decision process. Effectiveness 
was determined by a consensus opinion of the participants of the Concept 
Engineering research team and/or members of actual product development 
teams. The Plan-Do-Check-Act cycle is continuously applied to the 
development of the conceptual model and supporting methodologies. A 
brief description of each stage, as it currently exists, is provided below. Refer 


to Appendix A for more detailed information. 
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Concept Engineering 


1. Understanding Customer's Environment 


Step 1: Plan for Exploration 
Step 2: Collect the Voice of the Customer 
Step 3: Develop Common Image of Environment 


2. Converting Understanding into Requirements 


Step 4: Transform Voices into Requirements 
Step 5: Select Significant Requirements 
Step 6: Develop Insight into Requirements 


3. Operationalizing What You Have Learned 


Step 7: Develop and Administer Questionnaires 
Step 8: Generate Metrics for Requirements 
Step 9: Integrate Understanding 


4. Concept Generation 


Step 10: Decomposition 
Step 11: Idea Generation 
Step 12: Solution Generation 


5. Concept Selection 


Step 13: Solution Screening 
Step 14: Concept Selection 
Step 15: Reflection 





19 


Stage 1: Understanding Customer's Environment 

The objective of Stage 1 is for the development team to develop 
empathy for the customer, in the actual use environment of the product or 
service. Stage 1 consists of developing a plan for the team's exploration, 
doing the exploration, and using the data collected by the team to develop a 
contextual anchor from the images of the customers’ environment. The 
creation of this common mental map of the customer's environment is a 
unique aspect of Concept Engineering. 

The planning process begins with an articulation of the project scope. 
After the scope is outlined, appropriate market segments and customer types 
are identified for investigation. Prior to visiting the selected customers, the 
team members develop an open ended interview guide and interview skills. 
Pairs of team members (usually cross-functional) visit customers and conduct 
the interview at the customer's site and take verbatim notes of customer 
comments and their own observations. Upon completion of the interview 
process, images of the customer's use environment are selected and analyzed 
with a KJ diagram4 (Ofuji 1990, Kawakita 1991, Shiba et al. 1991a). This "Image 
KJ" is a link to the customer's real world and acts as a contextual anchor in 


the customer's environment for all future product concept decisions. 


Stage 2: Converting Understanding into Requirements 

Stage 2 distills what was learned from the customer exploration into a 
small set of well understood, critical customer requirements. In this stage, the 
Image KJ developed in Stage 1, is used as a contextual anchor in the 


development of requirement statements to ensure they are consistent with the 


4 kJ diagrams structure detailed language (vs. numerical) data into more general conclusions 
using semantic and abstraction guidelines. They are one of a family of tools invented by Jiro 
Kawakita and known as the KJ method (Kawakita 1991). 
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customers' environment. A small set of the vital few from the useful many 
requirements is selected and the relationships between them are analyzed. The 
process and guidelines for linking customer voices to contextual images and 
transforming the voices into requirement statements is unique to Concept 
Engineering. 

The transformation process converts the customer's language, often 
laden with emotion, into a customer requirement statement better suited for 
use in downstream development activities (Ofuji 1990). Each customer voice 
is explicitly linked to an image of the customer's environment and through a 
clearly defined process of successive refinement is developed into customer 
requirement statements/ ‘The entire team then selects the vital few customer 
requirements from the useful many through a democratic and iterative 
process® of identifying the most important requirements based on their 
respective understandings of the opportunity,/The KJ method (Shiba et al. 
1991a) is again employed to develop new insight and team consensus 


regarding the relationships among requirements. 


Stage 3: Operationalizing What You Have Learned 

In Stage 3, the goal is to ensure that the key customer requirements are 
clearly, concisely, and unambiguously communicated in measurable terms. 
The key customer requirements are validated with customers, operationally 
defined in measurable terms and the resulting information is displayed in 
such a way that the relationships between requirements, metrics and 
customer feedback is easily seen. The application of Kano's analysis, described 
in detail in Appendix A, to customer requirements is unique in US concept 


development activities (Kano et al. 1984). 


° The Multi-stage Picking-up Method, another of the KJ method tools, focuses on the most 
& & Up 
powerful statements by eliminating non-candidates in an iterative selection process. 
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Wp Two validation methods, self-stated importance assessments and 
Kano's analysis, are employed during this stage to assess customer attitudes 
towards the selected customer requirements. The self-stated importance 
questionnaire is a traditional marketing research technique (Griffin & Hauser 
1992) and indicates the relative importance of requirements. Kano's analysis 
(Kano et al. 1984) segregates requirements into four categories (Attractive, 
One-dimensional, Must-be, and Indifferent) depending on the relationship 
pe changes in functionality and changes in customer satisfaction. 

Additionally, i in Stage 3 the team develops and structures® metrics in order to 
| measure, quantitatively, requirement realization a This stage concludes with 
the development of a Quality Chart and Operational Definitions (Deming 
1986, Hauser & Clausing 1988, Juran 1988, Akao 1990) to integrate customer 


requirement understanding. 


Stage 4: Concept Generation 
This stage marks the transition in the development team’s thinking 

from the “requirement or problem space” to the “solution space.” In this 
stage the objective is to develop the largest number of potential solution ideas 
possible. Multiple perspectives of the development project are used to 
generate ideas from distinct vantage points. The use of a structured idea 
development process is uncommon in US product concept development. 

/The complex design problem is decomposed into smaller, independent 
sub-problems based on the customer's perspective and also from the 
engineering development perspective,/ The team creates, through individual 
and group collaboration efforts, an exhaustive list of ideas (both feasible and 


unfeasible) for each sub-problem; working first from the customer's vantage 


© Tree diagram method relates means to ends, which are in turn means to more general ends, ina 
hierachial relationship (Shiba 1991b). 


point before exploring the internal engineering perspective. Generated ideas are 
systematically reviewed and enhanced/ This stage concludes when each team 


member creates their ideal solution concept from the generated list of ideas. 


Stage 5: Concept Selection 

In the final stage of Concept Engineering a product concept is selected 
for downstream development. In this stage, concepts are systematically 
reviewed, compared, combined and enhanced in an iterative process of 
concept development. Concepts are evaluated against customer requirements 
and organizational/environmental constraints. 

In the previous stage, the development team generated a wide array of 
solutions to address collectively the set of customer requirements. /In this 
stage, the team thinks individually and together, seeks expert help, and 
experiments in the laboratory in an iterative process of combining and 
improving initial solution concepts to develop a small number of superior 
iencerts/ The "surviving" complete concepts are evaluated in detail against 
customer requirements and organizational constraints in order to select the 
dominant concept(s),/ When completed, an audit trail exists for tracing the 
entire decision process from project scope determination through detailed 


concept analysis as the Concept Engineering process is self-documenting. 


Decision Support Processes 

Decision Support Processes are designed to support decision makers in 
the various phases of problem solving. So far, no generalized problem 
solving process exists that has been empirically validated (Mintzberg et al. 
1976, DeSanctis & Gallupe 1987, Sainfort et al 1990). However, Mintzberg and 


colleagues in their classic field study (Mintzberg et al. 1976) of 25 strategic 
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(unstructured) decision processes concluded that the decision process has 
three phases: identification, development, and selection. Identification 
consists of both recognition and diagnosis routines. The development phase 
includes two basic routines: search and design. The selection phase consists of 
screening, evaluation-choice and authorization routines. Furthermore, the 
Mintzberg study identifies three sets of supporting routines: decision control, 
communication and political activities, which facilitate the three major 
phases of the decision process. 

In the context of the product concept decision process, Mintzberg et al. 
(1976) empirically developed problem solving process can be redefined to be: 
requirement identification, idea development, and concept selection. I will 
use this framework to illustrate how Concept Engineering is a complete 
Decision Support Process’, the table below outlines the relationships which 


are described in subsequent paragraphs. 


CE Step 


Identification Customer Selection Matrix 
- recognition Interview Guidelines 
- diagnosis Image KJ 
Transformation Process & 
Guidelines 
Multi-stage Picking-up Method 
Requirement KJ 


Development Self-stated Importance Assessment 
- search Kano's Analysis 
- design Multiple Design Decomposition's 
Idea Generation Process 
Solution Concept Generation 


Selection Screening Matrix 
- screen Selection Matrix 
- evaluation-choice Self-documenting Audit Trail 
- authorization 





’ This argument could also have been made with alternative descriptions of the problem 
solving process, i.e., Sainfort et al. 1990, MacKay et al. 1992. 
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Mintzberg et al. observed that the identification phase consists of both 
recognition and diagnosis activities. They defined diagnosis as "the tapping 
of existing channels and the opening of new ones to clarify and define the 
issues" (p.254). Concept Engineering provides conceptual and methodological 
guidance for clarifying and defining the issues. Stages 1 and 2 deal explicitly 
with exploring the market and converting the knowledge gained in the 
exploration into a well-defined and focused set of customer requirements. 
Specifically, in Stage 1, Understanding the Customer's Environment, a 
“Customer Selection Matrix" is developed to identify exploration arenas; this 
matrix explicitly includes past, present and prospective customers. Next, 
"Interview Guidelines" are developed to assist the focus of the exploration 
efforts. Stage 2, Converting Understanding into Customer Requirements, 
provides clear guidance in the form of "Translation Guidelines” and 
"Transformation Worksheets" for converting the Voice of the Customer 
information gathered in Stage 1 into unambiguous and nonrestrictive 
Customer Requirements Statements. The vital few requirement statements 
are identified using the Multi-stage Picking-up Method and structured using 
the KJ diagram (Kawakita 1991, Shiba et al. 1991a). 

The Development Phase observed by Mintzberg et al. consists of both 
search and design routines. They indicate four different kinds of search 
behavior: memory, passive, trap and active. In Stage 3, Operationally 
Defining Requirements for Downstream Development, the requirements 
developed and selected in Stage 2 are actively validated with potential 
customers through the use of Self-Stated-Importance questionnaires and 
Kano questionnaires. The Idea Generation step in Stage 4, Concept 
Generation, could conceivably incorporate all four types of search activities. 


The Mintzberg study identified design activities that result in either custom- 
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made or modified solutions. The concluding step of Stage 4 is the generation 
of custom-made solutions that address the set of customer requirements. It is 
possible that constraints imposed on the design team could limit idea 
generation, and thus solution generation, to existing solutions which would 
result in "modification" design activities. 

The Mintzberg study identified three routines in the Selection Phase: 
screen, evaluation-choice, and authorization. The first step in Stage 5, 
Concept Selection, is Solution Screening. In this step a "Screening Matrix” is 
employed to reduce the number of alternatives to a smaller number of 
feasible alternatives. Additionally, each proposed solution is evaluated 
against the customer requirements relative to a pre-selected datum. In the 
second step of Stage 5, Solution Selection, a more analytical comparative 
process is introduced, if necessary, to further assist the development team in 
identifying the dominant concepts. Authorization, the final routine observed 
by Mintzberg et al. in the Selection Phase is not specifically addressed in 
Concept Engineering. However, each step of Concept Engineering is self- 
documenting; some development teams have used their Concept 
Engineering working documents in their project proposal presentations 
before management authorization committees. 

The three routines that support the three central phases of the decision 
process observed by Mintzberg et al. are: decision control, communication, 
and politics. The decision control routine consisted of two basic activities: 
planning and switching. Decision planning consists of “a rough schedule for 
solution, a development strategy, and an estimate of the resources” (p.261). 
The Concept Engineering process is described with a flow chart outlining a 
coordinated set of conceptual steps. Furthermore, in the introduction to the 


Concept Engineering Manual (Burchill et al. 1992) a Gantt Chart displaying 
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the various activities and estimated completion times is provided to assist in 
project planning. Switching “directs the decision maker’s attention to the 
next step, to choosing the appropriate routine, such as diagnosis or search...” 
(p.261). With respect to switching, the Concept Engineering manual also 
provides checklists at the end of each step to assist in determining if a 
minimum set of observable conditions has been met before moving to the 
next set of activities. 

The Decision Communication routines observed by Mintzberg et al. 
include: exploration, investigation and dissemination. Exploration is 
described as a general or passive search for information. The investigative 
routine involves the focused search and research of special-purpose 
information. Dissemination involves communication of information about 
the decision process progress or outcome to ensure eventual acceptance. 
Concept Engineering is geared towards investigative information searches in 
that the objective and recommended information processing approach are 
clearly established for each step of the process. Concept Engineering facilitates 
dissemination by having clearly defined switching points and criteria and the 
self-documenting nature of the tools mentioned previously. 

According to Mintzberg et al. political activities “reflect the influence of 
individuals who seek to satisfy their personal and institutional needs by the 
decisions made in an organization” (p.262). This is consistent with Salancik 
and Pfeffer's (1974) view that power is used in organizations to influence 
decisions concerning the allocation of resources; the more scarce the resource 
the less objective criteria and the more power will be used in obtaining it. 
Additionally, Salancik and Pfeffer state that when there is a disagreement 
about the priorities and consequences of possible actions, decisions can not be 


rationalized. Hickson et al. (1971) propose that "preventive routinization" 
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reduces or removes uncertainty and thus reduces opportunities for the use of 
power. Concept Engineering, which assists all stages and supporting routines 
of the decision process, removes uncertainties, clarifies priorities and the 

relationships between potential actions and objectives. This in turn, increases 
the likelihood for rational decision making thus reducing the opportunity for 


political activities. 


Conclusion 

Concept Engineering is a conceptual model with supporting tools and 
techniques. Furthermore, in relation to the empirical decision structure 
outlined by Mintzberg et al. (1976), Concept Engineering addresses not only 
each phase of the product concept decision process (requirement 
identification, idea development, and concept selection) but also each of the 
supporting routines (decision control, communication, and politics). In these 
respects it should be considered a complete decision support process. 

The development of Concept Engineering, particularly given the active 
participation of corporate product development professionals, provided an 
opportunity to conduct a comparative study of product concept development 
activities. Chapter 2 outlines the research objectives, design and subsequent 
implementation. Chapter 3 presents Inductive System Diagrams as a method 
for developing and articulating grounded, substantive theories for the 
product concept decision process. Chapter 4 presents the evidence, inferences 
and propositions related to management choices in the product concept 
decision process. Chapter 5 outlines management diagnosis opportunities of 
product concept development. Chapter 6 summarizes the findings and 


outline potential future directions. 
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Chapter 2: Research Design 


This research was designed to develop a substantive theory to help 
clarify the product concept decision process and for generating data related to 
the effectiveness of Concept Engineering as a method for developing product 
concepts. A key ingredient for developing both the theory and the method 
was the use of comparison groups. By comparing similar and dissimilar 
groups we could more readily identify key concepts. However, I recognized 
that if some of the comparison groups were stacked in favor of success or 
failure, the conclusions reached could be misleading at worst and delayed at 
best. As a result, I requested randomization controls to address much of this 
bias to provide a stronger foundation upon which to build the method and 
theory. 

In the proposed design, each of three participating companies would 
identify two pairs of development teams. Each pair would be approximately 
similar in scope, demographics, and history. One team from each pair would 
be randomly assigned to use the Concept Engineering process while the other 
team would use Pugh's Concept Selection process (Pugh 1981) which is 
similar to Stage 5 of the Concept Engineering process. 

The progress and outcome of the research would be assessed in three 
ways. First, field research techniques for observations, interviews and survey 
instruments would be employed to develop an understanding of how and 
why Concept Engineering works. The specific questions pursued were 
expected to change over the course of the research, but based on an 
exploratory cause-and-effect diagram (figure 2.1) they would center around 
the concepts of: structured design process, clear customer requirements, and 


organizational commitment. 
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Second, objective process measures developed by the CQM Research 
Committee (CQM 1992) would be used at both the requirement generation 
and concept selection stages. Finally, subjective assessments by relevant 


company officers would be made of the performance of each team. 


Validity Threats 

Every research design should attempt to preclude as many validity 
threats as possible. Internal validity represents the degree of confidence that 
the input changes actually caused the observed outcomes. External validity 
reflects the degree of confidence that the results can be generalized to groups 
other than those studied. Construct validity addresses how well the research 
measures what was intended to be measured (Cook & Campbell 1979, Kidder 
& Judd 1986). The design outlined above, and agreed upon by representatives 
(a chief operating officer, a general manager and a director of product 
development) of the participating companies, attempted to minimize each of 


these validity threats. 


External Validity 

The nature of the companies participating in the study necessarily 
imposes some threats to external validity, or the ability to generalize the 
findings. Specifically, all of the Concept Engineering teams were fairly small, 
core teams of six to eight members; although the full development team 
could be substantially larger. Additionally, all participating companies 
considered themselves to be "high-tech" and were members of the Center for 
Quality Management (CQM). Membership in the COM requires a strong CEO 
commitment to Total Quality Management (TQM) and there is considerable 


training and emphasis on the use of many of the techniques (e.g. KJ diagrams, 
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Tree Diagrams) employed in Concept Engineering. Additionally, TQM 
emphasizes standardization and process orientations which may have 
affected the acceptance of the process by development team members. Finally, 
the design techniques introduced in this study represent only a small subset 
of the potential development process enhancements. These factors limit the 


ability to generalize the conclusions of the study. 


Construct Validity 

The principle of triangulation, well known in navigation, applies to 
research as well. At sea, any three measures of location taken by different 
methods, i.e. satellite versus radar, will position you at three different points 
on the map. Your true location is more likely to be within the triangle 
formed by the three points than exactly at any one point. Articles and text 
books on research methodology emphasize the importance of having 
multiple ways of measuring the constructs of interest (Cook & Campbell 1979, 
Kidder & Judd 1986, Blackburn 1987, Jick 1979). In the research design of this 
study, multiple assessment methods, some qualitative and some quantitative, 


were identified for use. 


Internal Validity 

This design was structured to address many potential threats to 
internal validity in order to increase the degree of confidence that the process 
changes actually caused observed changes in the outcomes, if any. 
Compensating Treatment 

Some threats to internal validity, which could be present in any study 
that supplies a favorable treatment to one group and not to the other, revolve 


around the reaction of the "no-treatment" group. On the one hand, they 


32 


could try harder than usual through a heightened sense of rivalry, and on 
the other they could become demoralized and not work as hard. An accepted 
device for addressing these threats is to provide the "no-treatment" group 
with some form of treatment (Cook & Campbell 1979, Kidder & Judd 1986, 
Blackburn 1987). In this study we intended to provide the non-Concept 
Engineering groups with Pugh's Concept Selection process (Pugh 1981). 
Pugh's process would be recognized as a development process enhancement 
in each of the participating companies. 
Experimenter Expectancies 

An additional threat to the validity of the study comes from the 
expectations of the person delivering the intervention. It has been shown 
that the administrator of the intervention can unwittingly bias the results 
provided by subjects (Cook & Campbell 1979, Kidder & Judd 1986). In this 
study, a graduate student was hired and trained as a research assistant to 
collect data on some of the teams. The research assistant was not provided 
with training in Concept Engineering and thus could provide a control 
against some forms of bias which may have been introduced by the author. 
Randomization 

The importance of randomizing treatment assignment was a critical 
component of the design's defense against the various threats to internal and 
external validity inherent in this study. Specifically, given the availability of 
concurrent, co-located control groups, many threats to validity, such as 
history (events that occur during the experiment unrelated to the treatment), 
testing (the impact of the measurement or observation process on the 
subjects), and instrumentation (the process of collecting data), could be 
compensated for through the design. However, the presence of a control 


group does not address the threat to validity from selection, the process used 
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to assign groups to a treatment condition. Without random assignment of 
conditions, the selection threat opens the door to a multitude of plausible 
rival hypotheses which could account for any observed differences. (Cook & 


Campbell 1979, Kidder & Judd 1986, Blackburn 1987) 


Research Implementation 

The actual implementation fell far short of the research design. While 
this clearly has implications for validation efforts originally designed for the 
study, it did not seriously disrupt the research objective of developing a 
grounded, substantive theory for the product concept decision process. 
However, the trials and tribulations experienced in this study are valuable in 
highlighting some of the difficulties associated with experimental research 
designs in organizational settings. 

All three companies that agreed to participate in the study in the fall of 
1991 sent representatives to the two-week training session in January 1992. 
One company (hereafter referred to as Company 1) began their first Concept 
Engineering effort in February 1992, the second company (Company 2) began 
their first Concept Engineering team in April 1992, and the third (Company 3) 
began in May 1992. It was immediately obvious that the first teams were not 
assigned on a random basis. In Companies 1 and 2 the appropriate managers 
selected an initial team they felt had a high likelihood of success. In 
Company 3, although it was not immediately apparent, the selection and 
staffing of the first team created a high likelihood of failure. This conclusion 
is validated by comments from senior managers in Companies 1 and 2 who 
specifically stated that they needed an initial success and in comments from a 
vice president of engineering in Company 3, who “felt" Concept Engineering 


was a ploy by Marketing to shift their work to Engineering. In short, random 
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assignment to address some of the traditional threats to validity (selection, 
maturation, etc.) did not take place in this study. 

The original design assumed that many teams would be working 
simultaneously; the calendar time associated with each team was expected to 
be approximately four months. In execution, each company started the first 
Concept Engineering effort and then waited for preliminary results before 
committing itself to support a second team. Furthermore, each team took 
about six calendar months to complete its work. This delay had enormous 
implications for the scope of the research given the available time of the 
primary researcher. In hindsight, it became clear that companies would be 
hesitant to commit a second team until the first team could be evaluated, at 
least provisionally. The length of time required for each team to complete its 
work was a surprise. The largest contributing factor to the increase in project 
time was delay time before starting. Once the decision was made for a team to 
apply Concept Engineering, several months might pass before meaningful 
effort was applied to the project. This was primarily due to other project 
commitments of team participants. This problem resulted in fewer Concept 
Engineering teams to be available for the study than had been intended in the 
design. 

With respect to the control groups, the first and second companies also 
provided a non-Concept Engineering comparison team in the spring of 1992. 
These teams were assigned on the basis of availability rather than on 
matching characteristics of scope, demographics, etc. In company 1, the 
comparison investigation was short-lived; the project literally exploded in the 
laboratory after two months. Subsequently, in Company 1, the division 
director was impressed enough with the results of the first Concept 


Engineering development team that he declared that all subsequent 
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sponsored development efforts would use Concept Engineering. In company 
2, the investigation of the comparison team (not well matched in scope) did 
proceed through to design approval, although the team did not use Pugh’s 
concept selection process. In company 3, the chief operating officer carried out 
an extensive study (an entire week, including the weekend, of his time) of 
Concept Engineering and declared that all company-sponsored development 
efforts would be required to use it in order to proceed through the company's 
Product Review Board process. As a result of these differences in approach in 
the three companies, the study of matched comparison groups called for in 
the research design did not materialize. 

Ultimately, the number and nature of cases investigated was 
significantly fewer than anticipated. Therefore, any attempts to evaluate the 
relative effectiveness of Concept Engineering are now subject to considerable 
threats from rival plausible hypotheses. However, I was able to observe 
extensively five development teams in four companies that used Concept 
Engineering and two development teams in two companies that did not. In 
addition, in Companies 1 and 3, it was possible to make historical 
comparisons with the previous project completed by development teams 
assigned to use Concept Engineering. For each development team studied, I 
typically attended every scheduled meeting, approximately 80 hours per team, 
and conducted two to three in-depth open-ended interviews with each 
member of the team and their managers; each interview lasting at least one 
hour. Therefore, although they lacked random assignment, the available 
teams did provide a rich comparative setting, with many similarities and 


dissimilarities, to explore for theory generation. 
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The Theory Generation Study 

In theory generation research, data collection and analysis are 
conducted concurrently (Glaser & Strauss 1967, Barton & Lazarsfeld 1969, 
Miles & Huberman 1984, Schein 1987). “Qualitative research in general and 
theory generation in particular, is essentially an investigative process, not 
unlike detective work. Observing one class of events calls for a comparison 
with a different class. Understanding one relationship reveals several facets 
which have to be teased out and studied individually. The theory is 
developed in large part by contrasting, comparing, replicating, cataloguing, 
and classifying the subject of the study” (Miles & Huberman 1984; p.37). 
Without joint data collection, coding, and analysis, the subtleties in the area 
of study, and opportunities to investigate them, can be lost. As a result, the 
evolving nature of desired information precludes the establishment of 
detailed, pre-specified sampling plans (Glaser & Strauss 1967, Barton & 
Lazarsfeld 1969). In the words of C.I. Lewis (1929) "Knowledge begins and 
ends in experience; but it does not end in the experience in which it began." 

Glaser and Strauss (1967) make an additional distinction between 
sampling required for theory development and theory verification. 
Theoretical sampling, sampling designed to develop rich comparative 
settings, is conducted to identify and investigate variables and their 
interrelationships in the development of theory. Statistical sampling is 
conducted to collect evidence to be used in descriptive or verification studies. 
As a result, they state that the researcher generating theory need not use 


random sampling techniques. 
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Participant Observation 

Forrester (1992; p.57) states the professional literature emphasizes how 
decisions should be made rather than how they are made and "there is not yet 
an adequate literature on what constitutes the practice of identifying decision- 
making policy." Forrester's observation on the general decision making 
process is consistent with specific research findings on the product 
development decision process which consistently identify large differences 
between product development theory and practice (Cooper & Klienschmidt 
1986, Gupta & Wilemon 1990, Mahajan & Wind 1992). In Forrester's view 
(1992; p. 52), "perceptive observation, searching discussions with persons 
making the decisions, study of already existing data, and examination of 
specific examples of decisions and actions all illuminate factors that influence 
decisions." 

"Technically a ‘qualitative observation’ identifies the presence or 
absence of something, in contrast to ‘quantitative observation,’ which 
involves the degree to which some feature is present" (Kirk & Miller 1990; 
p.9). Participant observers gather data in the daily life of the organization 
studied (Becker 1969). Two approaches to participant observation were 
combined in this research, grounded theory and action science. In grounded 
theory, the goal is to develop theory, intimately tied to the data, which 
explains, interprets and predicts what is happening in an area of investigation 
(Glaser & Strauss 1967; p.). The action scientist has as a goal, the development 
of theoretical constructs simple enough to be usable while enabling the actor 
to grasp all the relevant features of the situation (Argyris et al. 1985; p.78). 
This strategy has provided the opportunity to generate a substantive theory, 
intimately tied to actual practice, which provides insight and access to 


practitioners in the development of product/service concepts. 
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Action Science 

In action science, the emphasis is on obtaining or improving basic 
knowledge while solving practical problems.! Action science subscribes to 
the view that one of the best ways to understand the world is to try to change 
it. This perspective leads to a direct challenge of the normal science premise 
that the primary objective of science is to describe reality. However, action 
science does subscribe to several features of normal science including 
intersubjectively verifiable data, explicit inferences, disconfirmable 
propositions and public testing (Argyris et al. 1985). 

One of the primary goals of the action science perspective is to 
distinguish between espoused theories and theories-in-use. Espoused 
theories are those that an individual claims to follow. Theories-in-use are 
those that can be inferred from action. The objective is to make explicit the 
largely tacit propositional logic of the form “In situation s (as the actor 
constructs it), do a to achieve consequence c.” This means that the research 
must elicit data on what individuals actually say and do as they interact, as 
well as data on what they are thinking and feeling at the time of their actions 
(Argyris, et al. 1985; p.). The ability of the researcher to recognize the 
possibility for inconsistencies between theories-in-use and espoused theories 
depends in many respects on the familiarity of the researcher with the daily 
life of the organization; hence the importance of the participant observation 
research approach. 

The action science requirements for disconfirmable propositions and 
public testing ask that propositions or hypotheses be characterized by features 


that allow practitioners to disconfirm them. These include making 


' By analogy, this is similar to the early Operations Research work during World War II 
(Argyris, et al.. 1985). 
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propositions public, providing the directly observable data on which they are 
based, making them connectable to these data, and designing conditions 
conducive to testing them for validity (Argyris et al. 1985; p.233). Edgar 
Schein in his monograph, "The Clinical Perspective in Fieldwork" (Schein 
1987; p.29), states that the clinician typically starts with an “action research" 
model i.e. the assumption that one cannot understand a human system 
without trying to change it. As a result, the clinician learns about some of the 
most fundamental dynamics that operate in organizations, and the power of 
such work is its ability to provide better variables and better understanding of 
system dynamics than other research methods. This leads to the conclusion 
that perhaps the best use of clinical work may be in the construction of 


variables and theoretical models (Schein 1987, Blanck & Turner 1987). 


Grounded Theory 

Grounded theory approaches to generating hypotheses are 
characterized by the use of an exhaustive (and exhausting) data-coding and 
memo-writing regimen, aS well as the use of the constant comparison 
method of analysis. A grounded theory development process generally 
consists of the following activities: 
1) The researcher starts by coding each incident in his data for as many 
categories of analysis as possible. While coding an incident, the researcher 
attempts to compare this incident with all other incidents in the same 
category. 
2) The researcher regularly stops to record in "theoretical memos" his or her 


thoughts on the developing theory. 
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3) As the coding continues, the unit of comparative analysis changes from 
comparison of incident with incident to comparison of incident with the 
accumulated knowledge of the category. 

4) The accumulated knowledge is integrated into a unified whole. 

5) The theory is solidified as major modifications become fewer, non-essential 
categories are pruned, and higher level concepts are abstracted from the 
detailed categories previously developed from the data (Glaser 1965, Glaser & 
Strauss 1967, Glaser 1978, Strauss 1987). 


Constant Comparison Method 

In the constant comparison method, the objective of the sampling 
process is to allow for comparisons of differences and similarities among the 
units of analysis. This process of analyzing the similarities and differences 
produces the dense category development essential to well grounded theory 
generation. Minimizing differences among comparison groups increases the 
likelihood that a lot of information is available for developing of the basic 
properties and conditions of a category. Identifying similar data under 
comparison conditions of maximum differences identifies the fundamental 
explanatory variables. To integrate these variables into theory requires 
investigating the causes, consequences and constraints of these variables also 
under comparison conditions of maximized differences (Glaser & Strauss 


1967; p56-58). 


Variable Development 
One of the strengths of grounded theory methods is the coding process 
for category development (Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 


"The code conceptualizes the underlying pattern of a set of empirical 
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indicators within the data. Coding gets the analyst off the empirical level by 
fracturing the data, then conceptually grouping it into codes that then become 
the theory which explains what is happening in the data" (Glaser 1978; p.55). 
The process begins with "open-coding", a line by line analysis of the data 
which is diametrically opposite to the process of coding with preconceived 
codes. In open-coding the analyst attempts to code the data in as many 
different ways as possible. The analyst constantly looks for the "main theme", 
for what appears to be the main concern of or problem for the people in the 
setting (Strauss 1987; p.35). As the analyst's awareness of the central 
problem(s) emerges, they alternate open coding with very directed "axial 
coding". Axial coding consists of analysis done around one category at a time. 
As core variables begin to emerge, the analyst employs "selective coding" to 
focus coding to only those variables that relate to core variables in sufficiently 
significant ways to be used in parsimonious theory. In all 10 to 15 codes are 
typically enough for a monograph on a parsimonious substantive theory 


(Strauss 1987; p.32). 


Theory Generation Research Validity 

In verification research to test a hypothesis, the investigator must 
already know what it is they are going to discover (Kirk & Miller 1990). In 
theory generation research, by definition, the researcher does not know what 
they are going to discover. The relatively small sample sizes and lack of 
reliance on random sampling techniques associated with the theoretical 
sampling requirements of grounded theory methods generate conflict with 
many of the traditional tests of validity outlined by Cook and Campbell (1979). 
As a result, a fundamental issue of theory generation research is how to 


express the validity of the developed theories. 
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Glaser and Strauss (1967) discuss the four properties any grounded 
theory must have for practical application. The theory must fit the 
substantive area in which it will be used — the concepts and hypotheses 
supplied by the theory are closely tied to the data. Second, it must be readily 
understood by people in the area — it will make sense to the people working 
in the area. Third, it must be sufficiently general to be applicable in diverse 
situations — the level of abstraction must be sufficient to make a variety of 
situations understandable but not so abstract as to be meaningless. Finally, 
the theory must allow the user partial control over structure and process — 
the theory must contain sufficient concepts and their plausible interrelations 
to allow a person to produce and predict change. In short, the theory can be, 
and is, used by practitioners to guide what they do. 

Argyris, et al.. (1985) also propose four criteria for testing the validity of 
a theory. First is intersubjectively verifiable data — competent members of 
the scientific community should be able to agree at the level of observation, 
even if they disagree at the level of theory. The second criterion is explicit 
inferences — the logic that connects theory and observation should be 
explicit. Third is the use of disconfirmable propositions — the results of 
observations must relate to the acceptance or rejection of the theory. Finally 
is the concept of public testing — the users of a theory test its validity by 
comparing actual and predicted consequences following a change in their 
actions based on the research. 

From the Clinician’s perspective Schein (1987) states that the validity of a 
theory can be determined by its ability to predict the response to an intervention. 
The ethnographic view of validity emphasizes the issues of replication and 


internal consistency (Van Maanen 1983). 
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Walter Shewhart, the acknowledged developer of statistical process 
control, may have said it best when he wrote: “there is an important distinction 
between valid prediction in the sense of a prediction being true and valid 
knowledge in the sense of a prediction being justifiable upon the basis of 
available evidence and accepted rules of inference" (Shewhart 1938; p.). 
Shewhart (1938) points out that it is possible for predictions to be valid even 
when the knowledge supporting them is not. Similarly, valid inferences can be 
made from faulty evidence. Therefore, if theories result in testable predictions, 
then the validity of theory generation research can be judged on the basis of its 
evidence, inferences and predictions. 

Revisiting the validity criterion outlined above it would appear that 
Schein is concerned primarily with prediction. Van Maanen's concerns seem 
related to evidence and inferences. Glaser and Strauss appear to address 
evidence and inference but not prediction; in addition they are concerned with 
generalizability and user accessibility. Argyris, et al. appear to address evidence, 


inference and prediction. These observations are summarized in the table below. 


Glaser & Strauss Argyris, et al.. Van Maanen Shewhart 
Schein 


verifiable data 


Understanding explicit inferences | internal inferences 
consistenc 


propositions 
fee“ public testing 
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These three concepts: evidence, inferences and predictions, constitute a 





set of requirements which, if addressed in theory generation research, would 
allow researchers to observe and distinguish both the validity of the 


hypotheses (predictions) and the validity of the theory creation process 


4-1 


(evidence and inference). An important caveat is drawn from Kuhn's (1962) 
arguments on how paradigms affect our abilities to interpret the arguments of 
others, i.e. because we interpret issues from our paradigm not others, it will 
be difficult for distinct schools of thought to agree on whether any given piece 
of “knowledge” is valid because the accepted rules of inference are different. 
However, at a minimum, it should be possible to assess whether the 
hypothesis or prediction itself is valid. 

Finally, although not essential from a validity perspective, I would also 
encourage researchers to strive for user accessibility as a criterion for 
substantive theory construction. For practitioners to use a theory they must 
understand how variables under their control relate to the system of interest; 
if a theory describes a system without providing practitioners access it won't 


be used. 


Conclusion 

The original design had accommodations to many recognized threats 
to validity, but in the execution of the research the researcher was unable to 
ensure that the research was implemented as designed. A researcher simply 
does not have sufficient influence to control the operational requirements of 
organizations. On the other hand, the research project still provided a 
valuable opportunity to conduct detailed investigations of the development 
process in a comparative setting. This environment supports the goal of 
generating a substantive, grounded theory to provide leverage to 
practitioners in clarifying the product concept decision process. However, the 
methodologies appropriate to exploring this setting are not widely accepted as 


mainstream operations management (or social science) research methods. 
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Chapter 3: Inductive System Diagrams 


Social scientists, over the past sixty years, have developed 
methodologies to generate theory through an inductive process based on the 
intensive analysis of a small number of data sources. However, a difficulty 
associated with these, and most styles of qualitative theory development, is 
conveying credibility. The researcher usually presents only enough data to 
project credibility which is often not enough to allow for verification by 
others. A theory which is not clearly stated, and not well integrated, has a 
reduced likelihood of being accepted (Glaser 1965). 

Inductive System Diagrams combine aspects of Grounded Theory 
methods and System Dynamics. Grounded theory approaches are used to 
develop the variables which have a great deal of explanatory power and are 
intimately tied to the data. The cause and effect relationships among these 
variables are then shown using causal-loop diagramming techniques from 
System Dynamics. This combination of grounded theory and causal-loop 
diagramming allows researchers to generate and communicate substantive 


theories intimately tied to the data. 


Grounded Theory 

Grounded theory approaches to generating hypotheses are 
characterized by the use of an exhaustive (and exhausting) data-coding and 
memo-writing regimen, as well as the use of the constant comparison 
method of analysis. In the constant comparison method, the objective of the 
sampling process is to allow for comparisons of differences and similarities 
among the units of analysis. This process of analyzing the similarities and 


differences produces the dense variable development essential to well 
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grounded theory generation. Minimizing differences among comparison 
groups increases the likelihood that a lot of information is available for 
developing of the basic properties and conditions of a variable. Identifying 
similar data under comparison conditions of maximum differences identifies 
the fundamental explanatory variables. To integrate these variables into 
theory requires investigating the cause, consequences and constraints of these 
variables also under comparison conditions of maximized differences (Glaser 


& Strauss 1967; p56-58). 


Variable Development 

One of the strengths of grounded theory methods is the coding process 
for category development (Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 
"The code conceptualizes the underlying pattern of a set of empirical 
indicators within the data. Coding gets the analyst off the empirical level by 
fracturing the data, then conceptually grouping it into codes that then become 
the theory which explains what is happening in the data" (Glaser 1978; p.55). 
The process begins with "open-coding", a line by line analysis of the data 
where the analyst attempts to code the data in as many different ways as 
possible. As the analyst's awareness of the central problem(s) emerges, open 
coding alternates with very directed “axial coding". Axial coding consists of 
analysis done around one category ata time. As core variables begin to 
emerge, the analyst employs "selective coding" to focus coding to only those 
variables that relate to core variables in sufficiently significant ways to be used 
in parsimonious theory. In all 10 to 15 codes are typically enough for a 


parsimonious substantive theory (Strauss 1987; p.32). 


Open Coding 

By definition in theory generation research, the essential variables are 
not known; open coding is the start of the variable development process. 
During open coding each sentence is explored for as many possible concepts as 
possible. When coding the concept, it is assigned a variable name which is 
closely linked to the supporting data. Questions related to the occurrence of 
the concept are generated. These generative questions build sensitivity for 
future use in making comparisons when the next occurrence of the concept is 
encountered (Glasser 1978, Strauss 1987). An example, from my field notes is 


provided below: 


"This was a decision node in the conception of the 
product which was not made by systematic analysis.” - 
R&D manager 


Decision node. What is a decision node? How many are 
there? What are the necessary conditions for an event to 
be considered a decision node? Who initiates the 
decision? Who ratifies the decision? Who monitors 
them? 


Conception. When is a product conceived? What is the 
gestation period like? | can think of lots of analogies 
here, prenatal care, miscarriages, etc. ... 


Systematic Analysis. What constitutes systematic? 


unsystematic? When does one favor one over the other? 
Assuming systematic is preferred; how does one get 
away with unsystematic analysis? 


The open coding process generates a large number of variables quickly. 
Therefore it is necessary to reduce codes in use. The reduction occurs through 
a process of abstraction (Hayakawa 1990). In abstraction, variables which 


convey similar concepts are grouped together and a variable name, which 
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captures the essence of the common concept, is selected to be used in all 
references to this concept. In some cases, one code in the grouping represents 
the best label for the concept and it can be used for the variable name. In 
other cases, it is necessary to create a variable name which captures the 
common concept. In the example below, systematic analysis was selected as 


the variable name which best captured the common concept in all six codes. 


systematic analysis 
- design constraint tradeoff 
- performance comparisons 
- conscious dimension sacrifice 
- tradeoff equation 
- doing homework 


By investigating events under similar conditions, those concepts which 
are common in different settings represent the initial pool of potential 
explanatory variables. (It is highly probable that the final set of variables could 
be substantially different than the initial set.) Axial coding is used to develop 


better insight into these variables. 


Axial coding 

Axial coding represents an attempt to identify the causes, consequences and 
constraints of a variable under investigation. It is designed to build substantial 
knowledge about the selected variable and other variables it relates to (Glasser 1978, 
Strauss 1987). In studies where both participant observations and interviews are 
conducted, it can be very productive to conduct a "Causes, Consequences and 
Constraints" structured interview with participants as soon as possible after 
observation of the concept of interest. Reviewing existing field notes for evidence of 
causes, consequences and constraints can also be productive as the following 


example shows: 
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"Going back and doing the correlation effort yielded the same 
numbers and is documented. This gives us triple verification 
of what we are doing. So I'm willing to sign." - design engineer 


systematic Analysis causes Traceability causes Confidence 


Selective Coding 

When a variable begins to stand out as being the core category, as having 
extra-ordinary explanatory power, it is selected for focused coding. Coding 
activities are focused exclusively on the selected variable and the other 
variables with which it has significant relationships. All available data should 
be considered for review in selective coding (Glasser 1978, Strauss 1987). In this 
study, the KJ method (Kawakita 1991, Shiba et al. 1991a) was used to investigate 


core variable relationships, ( see Appendix D for examples.) 


Iteration 

Cycling back and forth between open, axial and selective coding occurs 
regularly early in the investigation and gradually decreases as the research 
progresses (Glaser & Strauss 1967, Strauss 1987). For example, at any time during 
this process insight regarding the variables or related inferences may occur. 
When this happens, immediately stop and write a "theoretical memo” before 
continuing or at a minimum make an appropriate annotation in the field notes 


as shown below (Glaser 1965, Glaser & Strauss 1967, Glaser 1978, Strauss 1987). 


"It is becoming increasingly important because this 
process is taking a long time, not just a long elapsed 
time because it is not calendar time, but in terms of 
people time it is extensive” - marketing manager 


systematic Analysis causes Labor Requirements. 
Memo: Labor Availability constrains Systematic Analysis 


oil 


The insight (captured in writing first) can trigger a change in coding strategy. 
In the example above, the data show evidence that Systematic Analysis causes 
Labor Requirements. A logical inference, not supported by the evidence, is 
that Labor Availability could constrain Systematic Analysis. Accordingly, 
additional theoretical sampling and/or more open coding connected to the 
concept of labor could follow from this insight. In another example, an 
integrating diagram can be developed on the basis of axial coding. Analysis of 
preliminary diagrams can (often) identify inferences regarding variable 
relationships which are not supported by available evidence. This can trigger 
additional theoretical sampling, open coding and/or axial coding as required 


to explore the proposed relationship. 


System Dynamics 

Forrester (1968; p.1-2) argues that a "structure (or theory) is essential if 
we are to effectively interrelate and interpret our observations in any field of 
knowledge." A hierarchical framework for identifying the structure of a 
system has been identified and developed in the system dynamics field (see 
for example: Forrester 1968, Goodman 1974, Randers 1980, Richardson and 
Pugh 1981). These principles of system dynamics can be applied to decision 
processes to develop their underlying structure (Forrester 1968, Goodman 
1974, Randers 1980, Sterman 1989). 

At their highest level, systems can be described as being open-loop or 
closed-loop (Forrester 1968). Forrester identifies open systems as being 
characterized by current performance is not influenced by past behavior; 
open-loop systems do not observe, and therefore react, to their own actions. 
Closed-loop systems, on the other hand, are characterized by the feedback 


from past performance influencing current actions. Decision processes are 
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closed-loop systems as they are imbedded in a feedback loop; the decision, 
based on the available information of the state or condition of the system, 
controls an action influencing the system condition, which generates new 
information, which is used to modify the next decision (Forrester 1968; p.4-4). 

Interconnecting feedback loops are the basic structural elements in 
systems which generate dynamic behavior (Forrester 1968, Goodman 1974). 
"Feedback loops are a closed path connecting in sequence a decision that 
controls action, the level of the system, and information about the level (or 
condition) of the system, the latter returning to the decision-making point" 
(Forrester 1968; p.1-7). However, at a lower level of hierarchy, feedback loops 
contain a substructure composed of two types of variables — levels and rates 
(Forrester 1968). The level (or state) variables describe the condition of the 
system at any particular time while the rate variables tell how fast the levels 
are changing (Forrester 1968). 

To illustrate these points, consider the decision process of filling a glass 
from a beer tap. When we are thirsty and the glass is empty, the decision is to 
open the tap fully. As the level of beer in the glass approaches the top, we 
decide to gradually close down the tap, reducing the rate at which beer enters 


the glass so that the tap is closed when the glass is full and no beer is wasted. 


Causal-loop Diagrams 

Causal-loop diagrams identify the principal feedback loops in a system 
without distinguishing between the nature, i.e. level or rate, of the 
interconnecting variables (Goodman 1974). Goodman (1974) outlines the 
steps of developing a causal-loop diagram as follows: 

1. establish the pairwise relationships of relevant variables; 


2. ascertain the polarity of the causal pairs; 


Se 


3. fit together the causal pairs into closed loops; and 

4. test for loop polarity. 

Through this process, the causal-loop diagram allows the analyst to integrate 
the variables they have developed, explicitly state the inferences they are 
making and clearly communicate their hypotheses regarding the dynamics 
associated with the structural relationships of the system. 

Pairwise variable relationships are diagrammed with directed arcs. 
Arcs are used to connect the factors which influence each other; the arrow 
indicating the direction of influence. Each arc is annotated with an indication 
of the causal change (polarity) between the two factors.! An "S" indicates that 
the two factors move in the same direction, i.e., all other things being equal, 
as one variable increases the other variable also increases. An "O" indicates 
that variables move in opposite directions, 1.e., all other things being equal, as 
one factor increases the other factor decreases. These pairwise arcs can then be 
connected to form feedback loops. 

There are two basic types of feedback loops, reinforcing (positive) and 
balancing (negative) feedback loops which are used to explain the dynamics of 
complex situations (Forrester 1968, Goodman 1974, Randers 1980). 

Reinforcing loops promote movement, either growth or decay, by 
compounding the change in one direction. Balancing loops resist change in 
one direction and try to bring a system back to a specified goal or equilibrium 
state. These two simple structures can be combined in an large variety of 


ways into casual loop diagrams which can be used describe complex systems. 


'Goodman (1974) uses '+' and ‘-' to indicate positive and negative polarity. Senge (1990) and Kim 
(1992) advocate the use of 'S' and 'O’. 
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ISD Step by Step Methodology 

The development of Inductive System Diagrams starts with identifying 
the central variables and concludes with mapping their relationship through 
causal loop diagrams. An modification of a step-by-step process for 
developing system diagrams developed by Burchill and Kim (Burchill & Kim 
1993) is outlined below: 
Step 1: Selecting a Variabie 

The focus of the investigation is established by identifying significant 
(core) variables (categories) and their symptoms. The initial selection of a 
variable is decided by its apparent explanatory ability or central importance in 
the events being studied. (This implies that considerable open coding and 
comparative analysis has been conducted by the researcher.) This can be done 
through axial coding — the process of specifying the varieties of conditions 
and consequences associated with the appearance of phenomenon referenced 
by the variable (Strauss 1987;64). 
Step 2: Identifying Causes and Consequences 

After a significant variable is identified, the next step is to identify 
other variables closely related to it. The data are analyzed to identify key 
factors which appear to drive or be driven by the selected variable. This can be 
accomplished by selective coding, wherein all other subordinate variables and 
their dimensions become systematically linked to the selected variable. 
(Strauss 1987) 
Step 3: Describe Factor Relationships 

After key factors associated with a variable have been identified, their 
interactions are diagrammed as causal-loop diagrams. The pairwise directed 


arcs developed during axial and selective coding are integrated into a closed 
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system. There are usually many variables to explore and it doesn't matter 
which one is selected first assuming all will be investigated. 
Step 4: Check Diagram Consistency 

The diagrams should be compared to the collected data to ensure they 
are grounded in the available facts. Often early diagrams contain links which 
are not supported by the presented evidence. If upon review, the researcher is 
confident the loop reflects the system dynamics, additional theoretical 
sampling or coding is necessary to ensure the theory remains "grounded" in 
the available data. Additionally, the diagrams should be investigated for 
“leaps of logic", i.e.. can the diagram describe the patterns of events without 
explanation. Finally, the diagram is reviewed to ensure factor labels are at the 
same level of abstraction (Hayakawa 1990). For example, "Design Constraint 
Tradeoff' and "Performance Comparison" would be at the same level of 
abstraction while the abstracted category, "Systematic Analysis" would be at a 
higher level of abstraction. 
Step 5: Integrating Causal-loop Diagrams into an Inductive System Diagram 

After all significant variables have been diagrammed, the individual 
causal-loop diagrams are combined to articulate the underlying structure or 
theory. A central theme is developed using a clearly dominant (core) variable 
or by linking variables which are common to multiple causal-loop diagrams. 
Remaining causal-loop diagrams are incorporated into the central theme. 
Variables may be combined and re-labeled at a higher level of abstraction 
(Hayakawa 1990). Additionally, low impact loops are eliminated to simplify 
the diagram. This integrated ISD is validated for logic flow, abstraction levels 


and consistency with the data. 
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Product Development Study Example: 

An example of the use of ISD in the development of a substantive 
theory for product development activities follows. The specific coding and 
analysis examples come from teams using the Concept Engineering method. 
All field notes were exhaustively coded and analyzed (an average of three 
hours of off-site effort for every hour of recorded notes) by the author and/or 
a research assistant. Additionally, much of the coding and analysis was 
reviewed by colleagues in a Field Research Methods Seminar. 

One team went from kick-off to product requirement determination in 
less than two months and on to final product concept selection in only two 
more months — considerably faster than historical performance. As a result, 
Development Time was selected for focused investigation (theoretical 
sampling/axial coding). Examples of relevant quotes from field notes (italics) 


are provided to illustrate the ISD process. 


“(On the previous project) This process would have provided a clearer 
vision!, a straighter path to the end result. I see the process saving time? by 


us 


eliminating missteps*.” - Engineering Development Manager 


Coding this statement for variable development might create categories 
for: 1) Design Vision Clarity, 2) Straighter Path, 3) Development Time, and 4) 
Missteps. Straighter Path and Missteps are conceptually similar and at a 
higher level of abstraction could both be dimensions of the category 


Misdirected Effort. These variables can be diagrammed as follows: 


Desi 
ee ___y  Misdirected__._._» Development 
Clarity O Effort S Time 


This diagram indicates that as Design Vision Clarity increases Misdirected 


Effort decreases causing Development Time to also decrease. 


D0 


The constant comparison method employed in a Grounded Theory 
approach requires that events be compared to other incidents in the same 
category. Accordingly, the following incident, from the same team, which relates 


to Development Time was compared to the instance above. 


“Someone that has buy-in! understands the how and why and can explain to 
other people horizontally or vertically. Along with buy-in is a belief or passion? . 
I think that where there is passion there is ownership and those two combined? ; 
when they exist in the same group of people and the team encounters problems 


ut 


they don’t last?. The team fixes it and moves on”.” - Marketing Product Manager 

Coding this statement for variable development might create categories for: 
1) Buy-In, 2) Design Objective Understanding, 3) Passion, 4) Ownership, 5) Design 
Problem Resolution and 6) Development Progress. To simplify coding, Buy-In, 
Passion, and Ownership can be combined into an abstracted category Conviction. 
Additionally, Design Objective Understanding is conceptually similar to the 
variable Design Vision Clarity in the diagram above and is abstracted into the 
variable Design Objective Clarity. Development Progress is conceptually similar 
to the variable Development Time; Development Time will continue to be used 
as it is less ambiguous then Development Progress. The resulting diagram, 


integrated with the previous diagram, is shown below: 


Misdirected 
pane a i 
Design S 
Objective Development 
Clarity ame 
bs S Design / 
Conviction S Peapleen 
Resolution 
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This diagram adds the conditions that Development Time decreases as 
Design Problem Resolution increases which in turn is driven by Conviction 
through Design Objective Clarity. The integrated diagram enhances the 
ability to compare future instances of Development Time with the 
accumulated knowledge by clearly and concisely displaying the current state 
of accumulated evidence and inferences. 

In comparing instances of Development Time from a second team at 
another company, using the Concept Engineering approach, an important 
difference was identified. This difference is exemplified by the following 
quotes: 

"Also, since we spent a lot of time with the requirement labels yesterday, 
perhaps we could shortcut a bit on the time without discussion and talk a 


little sooner.” Process Facilitator 


"We should generate (requirement metrics) in pairs, then bring the result to a 
vote. Why not skip the voting step in pairs and vote as a group.” Team 
Leader 

From these quotes a new category, Short-Cuts, can be derived. The 
second team, as a result of several disruptions in their project, planned to 
complete seven (of fifteen) steps of the Concept Engineering process in one 
week. Prior efforts, including the first team addressed above, allocated two to 
three weeks for these same activities. This caused the second team 
significant, self-imposed, time pressure. Time Pressure was also identified as 
a relevant variable relating to Development Time. A possible consequence of 
taking Short-Cuts can best be seen in one of the final comments during the 


second teams reflection period late Friday afternoon. 
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"Surprises me, that after all the discussion this week, some people don’t 
know what others are talking about. I should say everyone doesn't know 


what the others are talking about.” Development Engineer 


Adding the new categories, Short-Cuts and Time Pressure, to the diagram of 


accumulated knowledge above, results in the following diagram: 


Misdirected 
f° & 6 a -— 
Design 
Objective Development 
Clarity Time 
XN Short a de 
Cuts~e—____» Pressure 
7 ae x R2 
Design 
ico Problem 
Resolution 


This causal-loop diagram shows two reinforcing loops (R1 and R2) and 
one balancing loop (B). The reinforcing loops imply that increases in Design 
Objective Clarity can decrease Development Time and subsequently Time 
Pressure as a result of less Misdirected Effort and/or as a result of increased 
Conviction and Design Problem Resolution. The reduction in Time Pressure 
leads to decreased Short Cuts which increases Design Objective Clarity. The 
balancing loop implies that as Time Pressure increases Short Cuts also 
increase, thereby decreasing Time Pressure. However, Short Cuts also 
decrease Design Objective Clarity causing an increase in Misdirected Effort 


and a decrease in Conviction. This diagram can be continually validated as 
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additional instances of Development Time come to light; new variables will 
be added or relationships modified as dictated by the data. Eventually, 
modifications become fewer and a theory about Development Time, 


grounded in the data, can be clearly and concisely stated. 


Inductive System Diagram Reliability Assessment 

In the fall of 1992, seven Sloan School graduate students were 
presented with the Inductive System Diagram instructions and example 
presented above and extracts from my field notes, figure 3.2. The students 
ranged from Ph.D. candidates in System Dynamics to M.S. candidates with no 
prior exposure to System Dynamics. Each student independently prepared an 
Inductive System Diagram. In addition to providing final diagrams, many of 
the students also provided annotated transcripts, preliminary diagrams and 
the amount of time spent on the exercise. (Many of the individuals who 
indicated more time developed diagrams with fewer variables. I conclude 
from this, that some participants put more effort into the abstraction and 
simplification procedure described in step 5 of the Inductive System Diagram 
process. Therefore, a diagram with fewer variables and relationships may 


reflect a higher level of synthesis.) 
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The segments below come from an interview conducted with a member of a development team 
which was in the final stages of Concept Engineering. This team began in June 1992, spending an 
average of two days per week working together. This interview was conducted in September 
1992. The person being interviewed is the Engineering Manager for the business unit. 

~You've mentioned this was a process which was pretty well defined and handed to you, what is it 
about process that makes it appealing? 

Processes are nice just to know where you are. You need some sort of understanding to know 
where you are, where you want to be, are you heading in the night direction, sort of a navigational 
100] aeen:.: 

{Continue staying with process for a moment, what else is there? 

I think I still believe it is a good process, I see some potential benefits. It is going well, it is fun. I 
am almost tempted to say I am looking forward to it. It is one of the first times one of the TQM, 
activities has been enjoyable and fruitful. I haven't seen anything that indicates it is any less a 
potential solution to our problem. Although I have not had much contact with recent design 
projects, I don't believe we have ever gone out to customers without defining the product first. 
We haven't ever had so much discussion between engineers and marketing. We used to be 
sending a memo around, Engineering sent one, Marketing sent one, I don't think they read each 
others. I think all of this discussion ... 

~Can you describe what about the process makes you willing to spend so much time? 

I can say it the other way around, the negative, if I didn't think it was helpful I wouldn't spend so 
much time on it. 

¢What is the Impact of interaction between marketing and engineering? 

I hope it is gonna makes things go a whole lot smoother down the road. Once we do decide what 
we are going to do there shouldn't be any surprises. Having spent all this time with the marketing 
guy, those decisions will be made pretty quick . We have covered so much it will be hard to think 
we haven't thought about something. This is a social process that probably hadn't existed before. 
This is surely going to break down some barriers. They are both going to know where the 
definitions came from and that will make the resolution of conflict a lot easier and there will also be 
a lot of day to day contact between them. Traditionally the designer, in the 6 to 9 months that the 
designer does his thing, normally doesn't talk to the marketing guy. When he is done he just gives 
it to them and they say "hey, where is this or that" 

Why wont there be surprises? 

Communication issue. I hope the marketing and design guys will communicate after getting to 
know each other so well in this process. We have covered so many issues if something comes up 
they will say ya we covered that and will remember. Once we are done with this almost exhaustive 
idea generation. When we decide what to do we will understand a whole lot better what it is we 
are trying to do. It has been just recently that we have had a design document. Marketer writes the 
front page and the designer writes the back page and they don't read each others work. They are 
doing this together and will know where the roots are. | 

{What is the impact of knowing the roots? 

They are going to know how we got to where we are in the product definition. I would hope that it 
will keep us from going back and going off in a tangent. Once we get to that definition there 
should not be a whole lot of waffling, we should feel pretty comfortable about it. I don't think 
there will be a whole lot of questioning. Other aspect if designers do come across something we 
didn't expect they will recall we talked about that and will have a basis for discussing this with the 
marketing guys. It should be a pretty quick way for reevaluating the definition. It is always nice 
too know where you have come through. need to know where you want to go but need to know 
where you've been and where you are. 

What is the Impact of not waffling? 

the design should go a lot quicker. Designers wont get off on a tangent spending a week or two 
chasing something they think is neat, they will be more efficient. They will know what to solve 
and what not to solve. 


figure 3.2 
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Reliability Assessment Method 

Each diagram was quickly reviewed for conformance with basic system 
dynamic modeling requirements. One diagram was rejected from further 
consideration as the author (someone with no system dynamics exposure) 
duplicated the same variable in multiple loops rather than connecting the loops 
through a single expression of the variable. The remaining six diagrams were 
reviewed to assess: the degree to which the diagrams reflect similar variables, the 
degree to which variables are connected in similar sequence; and the degree to 
which the overall structure is similar. 
Variable Comparison 

To assess the degree to which the diagrams reflected similar variables, each 
variable was written on a separate slip of paper. Those variables which expressed a 
similar concept were grouped together, figure 3.3. If a grouping contained multiple 
variable names from the same diagram, the original diagram was reviewed to 
ensure consolidation of variables was consistent with the original drawing. For 
example, in diagram #3, the variables: Speediness of Decisions, Development Time, 
Effectiveness, and Development Progress could be consolidated under the concept of 
Product Definition Time without changing the structure of the original causal loop 
diagram. However, in diagram #2, combining the variables, Time Spent in Process 
and Perceived Progress in Project under the Product Definition Time concept would 
have been inconsistent as the author of Diagram #2 linked the variable Time Spent 
in Process to the interview statement: "...if I didn't think it was helpful I wouldn't 
spend so much time on it." After ensuring consistency, the group label was selected 
as the best exemplar of the concept in the grouping. Variables from each diagram 
which were not initially placed in a group were then reviewed to see if they could be 
added to an existing group, without changing diagram structure, to simplify 


analysis. 
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Figure 3.3 


* The author or Diagram 2 indicated that they wrote all variable names in a positive direction, i.e. the 
in-vivo codes of "missdirected effort" and "wasted effort" were abstracted into the category 
"effectiveness of effort" 


Directed Arc Comparison 

To assess the degree to which the diagrams reflected similar pairwise 
associations , each diagram was first redrawn annotating which variables in 
the original diagram would be consolidated under the exemplar (bold) 


identified above in lieu of the original variable labels, e.g. figure 3.4. 
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Figure 3.4 


Each diagram was subsequently redrawn using only the common variable 
names, e.g., figure 3.5. In redrawing the original diagram with new variable 
names the sign of the arc connecting two variables may need to be changed. 
For example, the author of diagram 2 explicitly chose to write all variable 


names in a positive orientation, i.e., the references to "misdirected effort" and 
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“wasted effort" in the text were abstracted into the variable "Effectiveness of 
Effort". However, the exemplar chosen for this category was Missteps. As a 
result, the relationship between Design Clarity and Effectiveness of Effort is 
reversed from that between Design Clarity and Missteps. Appendix B shows 


the analysis of all diagrams. 


Use of a pias 
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Figure 3.5 


Using the redrawn diagrams, with common variable names, each 
pairwise directed arc connecting two variables was reviewed and the 
relationship annotated in figure 3.6. A two digit code was utilized for audit 
purposes; the first digit represents the directional relationship and the second 


digit represents the source diagram number. 
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Figure 3.6 

Reliability Assessment Results 

A review of figure 3.3 shows all six diagrams reference the following 
variables: Product Definition Time, Level of Cross Functional 
Communication, Design Clarity, and Use of Concept Engineering. 
Additionally, all diagrams except diagram 5 also referenced the variables 
Support for Process and Missteps. Four diagrams also referenced the variables 
Ease of Conflict Resolution and Shared Understanding. Unfortunately, due 
to the varying amounts of time spent in developing the diagrams and the 
differences in modeling experience, I am unable to conclude if the differences 


in variable inclusion represent failures on the part of the authors to identify 
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the concepts or are the result of more effort at abstraction and simplification. 
However, a review of figure 3.6 indicates that all variables which were 
connected by more than one author showed a consistent relationship. 
Stepping back from the detailed level of analysis to review the basic 
structures identified by the authors also shows a high degree of consistency. 
All six authors show a direct relationship from Use of Concept Engineering to 
Level of Cross Functional Communication. Five of the six authors show a 
direct relationship from Level of Cross Functional Communication to Design 
Clarity and the sixth author shows an inverse relationship to Product 
Definition Time. Furthermore, four of the remaining five authors map a 
inverse relationship from Design Clarity to Product Definition Time usually 
via the intervening variable Missteps. All five authors who show a 
relationship from Product Definition Time indicate it has an inverse 
relationship either directly to the Use of Concept Engineering (1 diagram) or 
indirectly through the variable Support for Process (4 diagrams). In summary, 


all participants in the study identified the same basic structure, figure 3.7. 
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Increased Use of Concept Engineering causes increased Levels of Cross 
Functional Communication which results in increased Design Clarity which 
leads (via reduced Missteps) to a reduction in Product Definition Time which 
in turns leads to an increase (via Support for Process) in the Use of Concept 
Engineering. Furthermore, it should be noted that each variable generated 
from the data can be operationalized and the predicted cause and effect 
relationships tested. 

The results of this preliminary assessment of Inductive System 
Diagrams indicates that they appear to be reliable with respect to variable 
indentification and integration. However, a more complete test, involving 
more subjects and evaluators is required before a more definitive statement 


of reliability can be made. 


Causal-loop Diagram Limitations 

Causal-loop diagrams do not show the level and rate substructure of 
the system (Goodman 1974, Morecroft 1982, Richardson 1986). In cases 
involving rate-to-level pairwise directed arcs, the traditional definitions of 
positive and negative polarity fail because accumulation effects are lost 
(Richardson 1986). In the filling of the beer glass example used previously in 
this chapter, the link from the rate of beer flow to the level of beer in the glass 
fails the traditional definition: here a decrease in the rate of flow from the tap 
will not produce a decrease in the level of beer in the glass (Richardson 1986; 
p.160). As a result, accurate prediction of system behavior is difficult using 
only causal-loop diagrams and more detailed flow diagrams are required 
before developing simulation models (Goodman 1974, Morecroft 1982, 


Richardson 1986). 
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Wolstenholme (1982; p.547) makes a clear distinction between the 
system description (qualitative) analysis aspects of system dynamics and the 
simulation modeling (quantitative) techniques and states: "a good system 
diagram can formalize and communicate a modeler's mental image and 
hence understanding of a given situation in a way that the written language 
cannot." Coyle (1983; p.885) states that the difficult part of the operations 
research discipline is to clearly describe the interrelationships of the system 
under investigation and that system diagrams require "not much more than 
patience and persistence to apply ... in reaching a good first approximation to 
an adequate breadth of view in considering a complex problem." Goodman 
(1974) concludes that while causal-loop diagrams are insufficient for 
constructing simulation models they are useful for model conceptualization 


by organizing principal components and feedback loops. 


Conclusion 

Inductive System Diagrams have been introduced as a diagram-based 
method for systematic field-based hypothesis development and integration. 
Inductive System Diagrams build on the strengths of accepted coding practices 
for variable development. They can be used to integrate variable 
relationships and are easily modifiable as additional information becomes 
available. As a result, they facilitate the ability of researchers to use the | 
constant comparative method of analysis, an accepted approach for theory 
generation. The Inductive System Diagram method was found to have 
reliability in a small scale experiment involving experienced and novice 
dynamic model builders. Additionally, they allow for theory validity testing 
against the criteria of: verifiable data, explicit inferences and disconfirmable 


predictions. 
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Chapter 4: Time to Market Dynamics 


The expression "Time to Market" is composed of two distinct components, 
time and market.! In this comparative study, a fundamental difference was 
observed in the product concept development teams depending on whether they 
focused on TIME or MARKET in the expression Time to Market. After more than 
a year and a half of field observations, interviews, and analysis, key variables 
associated with the product concept development decision process and Time to 
Market dynamics have been identified using the inductive system diagram 
process, described in the previous chapter. In this chapter, I present an 
investigation of the causes, consequences and constraints associated with a focus 


on TIME or MARKET in the product concept development decision process. 


TIME to Market Orientation 

Decreased TIME to market has been identified as a key ingredient in 
successful new product development (Mansfield 1988, Takeuchi & Nonaka 1986, 
Gupta & Wilemon 1990). There are significant market share benefits to early 
market entrants (Urban et al. 1986) and considerable penalties for being late to 
market. For example, McKinsey and Company claims shipping a product six 
months late can reduce life cycle profits by one third in high growth, short life 
cycle markets (Reinertsen 1983). Additionally, competitive pressures are 
reducing product life cycles, further increasing the pressure to reduce product 
development time (Mansfield 1988, Schmenner 1988, von Braun 1990). 

Gold (1987) suggests several strategies to accelerate product and process 
development. These can be categorized as increasing reliance on external sources 


of innovation, increasing reliance on internal development of innovation, and 


'David Walden of BBN Inc. first pointed out this important distinction to me. 


a 


increasing reliance on innovative management approaches. Millson, Raj and 
Wilemon (1992) outline a general framework for reducing development cycle time 
that consists of simplification, delay elimination, step elimination, operation 
speedup and parallel processing. Mansfield (1988) shows that innovation time can 
be reduced by increasing innovation cost. Reinertsen (1983) states that in both high 
and low growth markets, over-running development costs 50% to meet schedule 
had only a 4% impact on life cycle profit before tax. Bower and Hout (1988) state 
that fast cycle capability is a management paradigm which shapes an organization's 
systems and attitudes around the simple concept that time is money. 

Based on the observations and analysis of my field research and 
supported by the above discussion, I propose the following proposition: 


P1: A TIME to market orientation increases pressure for progress. 


Time to MARKET Orientation 

Considerable research on new product development success highlights 
the central importance of understanding user needs, i.e., the market (see for 
example: Rothwell et al. 1974, Cooper & Kleinschmidt 1986, Pavia 1991). 
Houston (1986) states that customer focus, profits and organizational integration 
are frequently associated with the marketing concept and have become 
synonymous with having a customer orientation. Shapiro (1988) describes the 
characteristics of the market driven company to include widespread 
dissemination of important buying influence information, interfunctional 
decision making, and committed coordinated decisions. Narver and Slater (1990) 
state that marketing orientation consists of three behavioral components: 
customer orientation, competitor orientation, and interfunctional coordination. 
Kohli and Jaworski (1990) in an extensive review of the literature found three 


core themes related to market orientation: customer focus, coordinated 
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marketing and profitability. However, the results of their 62 field interviews, 
conducted with a diverse cross section of managers, found that managers felt 
profitability was a consequence not a condition of market orientation. 

Based on the observations and analysis of my field research and 
supported by the above discussion, I propose the following proposition: 

P2: A time to MARKET orientation increases customer oriented bias and 


functional integration. 


Comparison of Product Concept Development Teams 

A TIME to market team is one which attempts to specify the design 
objectives in an accelerated period of time. The team is under a great deal of 
pressure for progress and displays a willingness to make decisions with 
recognized data deficiencies in order to meet the (usually aggressive) development 
schedule. Participants orient their analysis of the issues to support their, often 
preconceived, perspective of the product concept. In the development team 
meetings, partisan behavior, in which individuals stake out positions and 
vigorously defend them, dominates. The engineers discuss product attributes 
from the perspective of technology opportunities and constraints. Marketers 
discuss product attributes with respect to market segments and competitors. 
Although both groups are at the same meeting, they don't participate in the same 
process: the language used is different, the relative emphasis on product attributes 
is often different, and individuals can be observed to periodically disengage from 
the decision process based on discussion subject matter. Product concept decisions 
are ultimately made, but it is difficult for the entire team to recreate and defend the 
decision choices to the management review board. When all is said and done, one 
or more groups lack commitment to the product concept and there is a high 


expectation that the final product will differ from the initial concept. 
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A time to MARKET team is one which attempts to develop credible design 
objectives which reflect a deep appreciation of the customer's requirements. The 
team is characterized by decision analysis oriented to maximize customer benefit. 
In development team meetings, every individual participates in all aspects of the 
decision process. Members frequently put their statements in the context of specific 
customer encounters to clarify or emphasize their positions. Relevant issues and 
information regarding design objectives are considered to everyone's satisfaction 
before the team moves on to subsequent development activities. This cross- 
functional collaboration to create a common appreciation of the design objectives is 
apparent when the team presents the product concept to the management review 
board. All team members display a commitment to the product concept and can 
credibly trace their decision process when required to justify their choices. 

The observed variable relationships are outlined in the table below. 

time to MARKET 
orientation orientation 
Decision Variables |S 


Pressure for Progress 

Systematic Concept Analysis 
Prejudiced Perspective 
Functional Integration 
Analysis Depth 


Supporting Evidence 


Contextual Awareness 
Process Participation 
Traceabili 


Design Objective Appreciation 
Requirement Clarity Lower 
Requirement Credibility Lower 
Substantive Progress 
Concept Commitment Lower 
Misdirected Effort Higher 
[Constraints SC 
Labor-hour Requirement 
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The dynamics of a TIME versus MARKET orientation in the expression Time- 
to-Market may be easier to understand by representing the data in the table above as a 


high level inductive system diagram (see Chapter 3). 


2 
MARKET 
> Orientation aie 
Presb —_ oF Systematic 
Concept 
— 6 Analysis 
S Concept § 
Development 
Development Se _S 
ae Supporting 
Evidence 
O A / 
Substantive Design S 
Accomplishments Objective 
S = Credibility 


A relative emphasis on TIME increases Pressure for Progress and reduces the 
opportunity for Systematic Concept Analysis. This reduction in systematic analysis 
decreases the labor requirement and consequently the concept development time. 
However, it also decreases the Supporting Evidence needed to justify concept 
decision choices. The resulting reduction in Design Objective Appreciation 
subsequently reduces Substantive Accomplishments as time and resources are spent 
on tangents and detours in downstream development efforts. The net result is 
increased Development Time and increased Pressure for Progress. 

A MARKET orientation decreases Pressure for Progress, relative to the TIME 
oriented development teams, and increases Systematic Concept Analysis. The 


increase in Systematic Concept Analysis increases Supporting Evidence, but also 
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increases the labor requirement and Concept Development Time. However, the 
resulting increase in Design Objective Appreciation focuses development efforts 
thereby increasing Substantive Accomplishments which in turn will decrease 
Development Time and Pressure for Progress. 

The dynamics described above represent a classic "Fixes that Fail" archetype 
(Senge 1990) in which the unintended consequence of a problem solution over time 
contributes to the problem it was trying to solve. In this case the emphasis on 
reducing TIME to market decreases concept development time but inadvertentdly 
reduces design objective appreciation resulting in waste and rework in downstream 
development activities increasing total development time. On the other hand, the 
fundamental solution, an emphasis on MARKET, actually reduces total time by 


saving the time spent on misdirected downstream development efforts. 


Presentation Structure 

The presentation of the underlying variables identified in this study follows 
the order of the left-hand column of the concept decision variable table in the 
previous section. (This order is also a clockwise rotation around the inductive 
system diagram presented above.) The decision variables observed in the concept 
development process include pressure for progress and systematic concept 
development (prejudiced perspective, functional integration, and analysis depth). 
The objective function of product concept development activities was observed to be 
the maximization of supporting evidence (contextual awareness, process 
participation, and traceability), design objective appreciation (requirement clarity 
and credibility) and substantive development accomplishments (commitment and 
misdirected effort). The primary constraint observed was available resources, 
specifically labor-hours. The table below provides a brief definition of each variable 


and indicates the page which contains an expanded variable description. The 
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expanded description includes evidence in the form of representative quotes, a 
diagram of the inference from the evidence and a propositional statement of the 
prediction from the inference. Following the expanded definitions, the variables are 
integrated into an inductive system diagram. A discussion of conclusions, 


contingencies and rival plausible hypotheses follows the integrated inductive system 


diagram. 
VariableName [Page] Definition SS 













Pressure for Progress An implicit or explicit force on the 
development team to proceed rapidly 
through development activities. 


The formation of an opinion on the product 
concept without knowing all the facts. 
groups, primarily marketing and engineering. 








































place a requirement statement in the context 
Process Participation 16 | The active involvement of participants in the 

The capability to recreate decision choices 

requirements and their relative priorities are 
Credibility 20 

information related to design objectives. 


Analysis Depth 13 | The degree of investigative thoroughness 
associated with concept development 
decisions. 
of the customer's environment. 
complete (requirement identification, idea 

Traceability 7, 
with the supporting justification. 
identified and agreed upon. 

Concept Commitment 21 | The level of support the concept has earned 


Contextual Awareness The ability of a development team member to 
development, and concept selection) 
Requirement Clarity 18 | A product definition in which the vital few 
The perceived validity and accuracy of 
from the development team and their 

















Misdirected Effort Development activities resulting in detours, 
tangents and missteps relative to the design 


objectives. 


Labor Requirement Gap | 23 | The difference between required and 
available labor for concept development. 










Decision Variables 

In this study the decision variables observed in the concept development 
problem include pressure for progress and systematic concept development. 
Pressure for progress was either an explicit or implicit force on the development 
team to rapidly proceed through development activities. Systematic concept 
development consists of three components: prejudiced perspective, functional 
integration, and analysis depth. Prejudiced Perspective indicates the formation 
of an opinion on the product concept without knowing all the facts; it was 
observed as biased decision making by internal stakeholders. Functional 
Integration reflects the degree of interaction between various functional groups 
in the stages of concept development; cross functional interaction was observed 
to range from collaborative to partisan. Analysis depth represents the degree of 
overall thoroughness of analysis associated with the concept development 
process; investigations were observed to vary from comprehensive to 
incomplete. It was observed that each of these variables was capable of being 


influenced by management to a greater or lesser, but always non-trivial, extent. 


Pressure for Progress 

Several researchers indicate that the pressure for accelerated new product 
development can cause development organizations to conduct a less than 
thorough job in order to have the appearance of progress (Van de Ven 1986, 
Gupta & Wilemon 1990). Contributing to this phenomenon may be the 
disconnect between product development theory and practice. Bower and Hout 
(1988) specifically emphasize the requirement that fast cycle companies have 
development processes which are well defined and understood. They state that 
in addition to visibility of the process from start to finish, the fast cycle 


organization understands the interrelationships of the process components and 
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organizational policies. The strategies outlined by Millson, et al. (1992), e.g. step 
elimination, delay elimination and parallel processing, all require development 
process understanding. However, the product concept decision process is not 
well defined or understood. The observed disconnect between the requirement 
for process understanding in theory and the lack of process knowledge in 
practice is consistent with other studies of product development theory and 
practice which indicate that what the literature recommends and what the 
actually happens are significantly different (Cooper & Kleinschmidt 1986, Gupta 
& Wilemon 1990, Mahajan & Wind 1992). 

In this study, Pressure for Progress was observed to lead to decision 
process speedup. When pressured for progress participants were observed to 
display self-interest oriented behavior based on a prejudiced perspective. 
Pressure for Progress was also observed to lead to a willingness to make 
decisions with data deficiencies. Pressure for Progress was observed to dominate 
other decision variables specifically causing stakeholder orientation and 
incomplete analysis. 


Evidence, inferences and propositions are provided below. 


"Some of the features that make the applications engineer job easier, I wouldn't have 
thought about this if I was under pressure; I wouldn't have put them in my design.” - 
design engineer 


Pressure Stakeholder 
for ———® Prejudiced 
Progress Perspectives 


P3: As Pressure for Progress increases Prejudiced Perspectives increase. 
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"You need to be very quick to get something up on the screen. In the initial stages you 
don't want to get all hung up on all of the goals” - design engineer 


Pressure I eh 
ey: . ncomp e e 
P S Analysis 
rogress 


P4: As Pressure for Progress increases Analysis Depth decreases. 


Prejudiced Perspectives 

Van de Ven (1986) claims the first, of four, central problems in managing 
innovation involves the problem of managing human attention; overcoming 
individuals’, and organizations’, natural tendency to be focused on and 
protective of, existing practices. The self-interested behavior of individuals (or 
groups) in settings which require interdependencies can lead to conflict (Kohli & 
Jaworski 1990, Ancona & Caldwell 1992). Self-interested behavior and conflict 
lead to reduced cross-functional integration (Gupta et al. 1986, Souder 1988, 
Kohli & Jaworski 1990, Ancona & Caldwell 1992). 

Dougherty (1992) suggests that viewing the product from the user's 
perspective can provide a basis for developing a common understanding of the 
desired innovation. Developing the user's perspective involves more than 
obtaining the customer's verbalized needs and preferences; it includes analysis of 
the factors which influence those needs and preferences (Kohli & Jaworski 1990). 
Placing design engineers in direct contact with customers provides an 
opportunity for developing this deeper level of understanding (Rothwell et al. 
1974, Kohli & Jaworski 1990, Bailetti & Guild 1991). 

In this study, analysis was observed to occur from either a customer or 
stakeholder perspective. Customer oriented perspective is present when the 


concept development decision process biases innovation efforts towards 
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customer benefits. Customer orientation was most often evident by the use of 
numerous, specific, references to customers during all phases of the design 
objective decision process. In contrast to customer orientation is stakeholder 
orientation. Stakeholder oriented perspective is evident when the decision 
process is biased towards satisfying the prejudiced opinions of people in 
positions of power (on the team or in management) who often dictate product 
design objectives. In this study, it was observed that Customer Orientation 
increased Contextual Awareness and Crossfunctional Collaboration. 


Evidence, inferences and propositions are provided below. 


"Getting oriented to customer needs, in some cases changing biases and getting 
intimately familiar with customer requirements and environments.” - team leader 


Prejudiced Contextual 
Perspectives OQ Awareness 


P5: As Prejudiced Perspectives decrease Contextual Awareness increases. 


"If development and marketing hear more of the same stuff from customers it has to build 
a better working relationship.” - team leader 


Prejudiced Crossfunctional 
Perspectives O Collaboration 


P6: As Prejudiced Perspectives decrease Functional Integration increases. 


Functional Integration 

Lawrence & Lorsh (1967; p.11) define integration as the "quality of the 
state of collaboration that exists among departments that are required to achieve 
unity of effort by the demands of the environment". Functional integration has 
been shown to be a key factor in successful innovation (Rothwell et al. 1974, 


Gupta et al. 1986, Gupta & Wilemon 1988, Pinto & Pinto 1990, Moenaert & 
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Souder 1990, Dougherty 1992, Song & Parry 1992). Innovation is essentially an 
information based activity and as a result information transfer is the major 
integration vehicle (Moenaert & Souder 1990). Communication effectiveness has 
also been linked with innovation success (Rothwell et al. 1974, Ancona & 
Caldwell 1992). However, achieving effective communication and integration is 
difficult; nearly 2/3 of the 289 new product development efforts studied by 
Souder (1988) experienced some degree of interface disharmony between R&D 
and marketing. 

In this study, all of the companies observed had a practice of assigning 
multiple functions, marketing and engineering at a minimum, to product concept 
development teams. However, the degree of interaction among the various 
functional groups on the development team was observed to range from 
collaborative to partisanship. Crossfunctional collaboration is the ability of 
different functional groups to work together in the concept development decision 
process. In collaboration, a synthesis of the perspectives of the functional 
representatives assigned to the team is incorporated into the concept development 
decision process. In contrast to crossfunctional collaboration is functional 
partisanship: the advocation or strong support of one particular perspective. In 
this study, crossfunctional collaboration led to process participation. Functional 
Partisanship was observed to lead to incomplete analysis. 


Evidence, inference and propositions are provided below. 


"If designers do come across something we didn’t expect they will recall we talked about 
that and will have a basis for discussing this with the marketing guys. It should be a 
pretty quick way for reevaluating the product definition.” - engineering manager 


Crossfunctional Process 
Collaboration Participation 


P7: As Functional Integration increases Process Participation increases. 
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"The designer and marketing guy go to the field and get information. Interpretation of 
the data and how to apply it to a product is left up to the individual and people interpret 
differently based on their backgrounds” - design engineer 


Functional Incomplete 
Partisanship S Analysis 


P8: As Functional Integration decreases Analysis Depth decreases. 


Analysis Depth 

Several studies indicate that innovation success is significantly impacted 
by the number of product development process steps completed; the more 
thorough the job the more likely the success (Cooper & Kleinschmidt 1986, Gupta 
& Wilemon 1990, Wilson 1990, Mahajan & Wind 1992). Unfortunately, Cooper 
and Kleinschmidt (1986) found that "up front" new product development 
activities were more likely to be performed poorly or not at all. Additionally, 
Moenaert and Souder (1990) indicate that industrial new product development 
success is related to the effectiveness of information processing. However, Gupta 
& Wilemon (1990) found 47% of the participants in their study were frustrated by 
the lack of attention to detail during product development. 

In this study, the analysis associated with product concept development 
was observed to range from comprehensive to incomplete. Comprehensive 
analysis entails a thorough identification of key design requirements, an 
extensive development of alternative ideas and a systematic concept selection 
process. In contrast to comprehensive analysis is incomplete analysis. 
Incomplete analysis is characterized by decisions with both recognized and 
unrecognized data deficiencies and resulting premature switching between 
decision process stages. In this study, it was observed that Comprehensive 


Analysis leads to traceability whereas Incomplete Analysis does not. 
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Evidence, inference and propositions are provided below: 


Marketing Rep: “Lets move on to the competition comparison sheet; did you get .1%? 
Engineering Rep: "No I didn't; but of course I'd be better.” 

Marketing Rep: “Any idea?” 

Engineering Rep: "Totally a guess; maybe 10 instead of 15.” 

Marketing Rep: “I'll leave it blank.” 

Observer: Product definition presentation to management one week later listed 10 for this 
specification. 


"When I use ‘systematic thinking’ in this I think here is what the customer said and here 
is the requirement which matches what he said.” - marketing manager 


Analysis ~ 
peer SA Traceability 


P9: As Analysis Depth increases Traceability increases. 


Objective Function — Supporting Evidence 

Supporting Evidence provides participants the ability to justify the 
decisions made during the entire concept development decision process. There 
were three observed conditions associated with supporting evidence: contextual 
awareness, process participation and decision traceability. In the time to 
MARKET teams, references to customer contexts were used to clarify issues 
throughout the entire concept development decision process. Participation in the 
decision process allows the design team member to know the background on 
how the design objective decisions were reached. Evidence provides participants 
a way to confidently recreate the decision process should/ when the need arise in 


the future. 


Contextual Awareness 

Moenaert and Souder (1990; p.221) state: "A variable that has been seldom 
addressed in previous research, and which seems to be essential to the use of 
information received from other functional fields, concerns the contextuality of 
the received information. The contextuality of information refers to the degree to 
which the source has given the receiver the necessary information and references 
such that the receiver can see the relevance of this information for his/her work 
on a particular project." They conclude that extrafunctional information, without 
supporting context, cannot be processed and used. 

In this study, Contextual Awareness of the customer's requirements was 
observed to increase the clarity of the design objective. Contextual awareness is the 
ability of development team members to place a requirement statement in the 
context of the customer's environment. Written requirement statements ideally 
represent a high fidelity translation of the customer's actual needs. However, even 
in the best processes for capturing the voice of the customer, the written requirement 
statement lacks the affective qualities of an actual customer interaction and is subject 
to different interpretations. Placing requirement statements in the context of the 
customer's use environment clarifies the intent of the requirement statement. In this 
study, it was observed that stories of real customer experiences, whether obtained 
specifically in efforts associated with the current project or in other efforts, were 
used by development team members to clarify written requirement statements. 


Evidence, inferences and propositions are provided below. 


"The team began a discussion of what a requirement statement meant. Eventually they 
went back to the interview transcript and rewrote the requirement statement.” - observer 


Contextual Requirement 
Awareness S Clarity 


P10: As Contextual Awareness increases Requirement Clarity increases. 
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Process Participation 

Several studies indicate that interaction between producers and users of 
market information significantly impacts the credibility and utilization of the 
information in the innovation process (Deshpande & Zaltman 1982, Deshpande & 
Zaltman 1984, John & Martin 1984). Gupta and Wilemon (1988) state that credibility 
has two main components — information credibility and source credibility. 
Deshpande and Zaltman (1982) conclude that personal interaction increases trust in 
the source and consequently the content of the research. Additionally, 
crossfunctional participation in market research increases the effectiveness of 
information exchange between functions (Deshpande & Zaltman 1984, Kohli & 
Jaworski 1990). John and Martin (1984) found that the proximity of participants to 
the marketing planning activities enhanced credibility which they attribute to 
communication facilitation. 

In this study, Process Participation was observed to build design objective 
credibility. Process participation represents the active invo:vement of participants in 
the complete decision process. This implies that individuals participate in 
requirement identification, idea development and concept selection activities. This 
is contrasted with event participation in which individuals participate in some 
events and not others, i.e. participation in concept selection but not requirement 
identification activities. Participation in all stages of the design objective decision 
process provided team members with a common understanding of the events which 
led to the concept selection; this increases requirement clarity and credibility. 


Evidence, inferences and propositions are provided below. 


"They (marketing and engineering) are doing this together and will know where the roots 
are. They are going to know how we got to where we are in the product definition.” - 
engineering manager 


Process > Requirement 
Participation S Clarity 


P11: As Process Participation increases Requirement Clarity increases. 


"There was engineers in the process. It wasn't marketing pukes saying this is what it is. 
When we got into the room there was built in credibility because the person who is 
processing the information is witnessing data collection.” - marketing manager 


Process > Credibility 


Participation 


P12: As Process Participation increases Credibility increases. 


Traceability 

Van de Ven (1986) states that the legitimacy of the decision process is the 
dominant evaluation criteria used to assess innovative ideas as the ideas 
themselves can rarely be judged. Deshpande and Zaltman (1982) found that 
managers were inclined to pay critical attention to research methodology when 
the findings were surprising. Moenaert and Souder (1990) found that 
information which was not formally substantiated by convincing evidence was 
less likely to be used. They also observe that a disadvantage of face-to-face 
discussions is the absence of hard copy which can be used in the future to justify 
actions taken. 

In this study, Traceability was observed as a highly desirable outcome for 
justifying the decisions made during the product concept development process. 
Traceability includes, but is not limited to, documentation of the outcomes of the 


decision process. Just as importantly, traceability regarding the concept 
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development decision process itself provides credibility to the identification, 
development and selection of design objectives. In this study, it was observed 
that the ability to document the decision process increased the credibility of the 
design objective decisions. 


Evidence, inferences and propositions are provided below. 


"I have higher confidence and better ability to sell to executive committee because the 
process is self-documenting.” - marketing manager 


Traceability “Seaton Credibility 


P13: As Traceability increases Credibility increases. 


Objective Function — Design Objective Appreciation 

Design objective appreciation occurs when the development team has 
discriminating perception and confidence in a clearly defined set of 
requirements. Requirement clarity results from understanding the vital few 
requirements and the relative priorities within this set of requirements. Design 
activities fundamentally involve making tradeoffs; design objective appreciation 
facilitates tradeoff optimization. The development team needs to clearly 
understand the subset of the total universe of requirements which will be 
emphasized in the product concept to ensure effort is focused effectively. 
Requirements are assessed for credibility, by the development team, in order for 


them to have confidence in the relative merit of the tradeoff decisions. 


Requirement Clarity 
Understanding user needs has long been recognized as a significant factor 
to new product development success (Rothwell et al. 1974, Cooper & 


Kleinschmidt 1986). The absence of a clear product definition has been linked to 


instability in product and marketing plans (Gupta & Wilemon 1990, Wilson 
1990). Gupta and Wilemon (1990) found poor definition of product requirements 
was the reason most cited for product development delays. 

In this study, Requirement Clarity was observed to consist of two 
components: the identification and agreement on the vital few requirements and 
their relative priorities. The establishment of the vital few requirements focused 
development efforts. In all observed projects, the development team attempted 
to identify a small subset of the total requirements which would differentiate the 
proposed product from the competition (either internal or external). Knowing 
the relative importance of these key requirements assists in making development 
tradeoffs. The relative requirement priorities clarify what to work on and, just as 
importantly, what not to work on. There was a concerted effort on each observed 
development team to clearly establish the relative priorities of acknowledged 
design objectives. The product definition, by focusing on only the vital few 
requirements, is designed to serve as a road map for the development team, 
providing direction and flexibility for making tradeoffs while minimizing 
unproductive effort. 


Evidence, inferences and propositions are provided below: 
"Documented lists of the most important things to be concerned with ... all too often you 
put blinders on, getting in a rut, attacking one piece of the problem and let other aspects 
slide.” - design engineer 


"The implication of having visibility (of customer requirement priorities) is it tells you 
what is good enough, where you have to put efforts or where you can compromise, not 
spend time." - design engineer 


i Misdirected 
Requirement 
ae — > Development 
O Effort 


P14: As requirement clarity increases misdirected development effort 


decreases. 
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Credibility 

John and Martin (1984) define credibility as a composite of six components 
related to the perceived quality of the marketing plan; these components assess 
whether the plan is: realistic, accurate, specific, consistent, complete, and valid. 
Gupta and Wilemon (1988) conclude that in organizations with a high degree of 
integration between marketing and R&D, R&D personnel perceive marketing 
information to be: realistic and valid, objective, consistent and complete, useful 
and appealing. Moenaert and Souder (1990) define credibility as a measure of 
the degree to which the receiver believes the information to be undistorted and 
state that the credibility of received information will be a positive function of: 
validity, accuracy and clarity and a negative function of surprise. Deshpande & 
Zaltman (1982) focus less on "credibility" per se but rather on the instrumental 
use of knowledge; it is the knowledge applied to a particular decision. They 
identify four dimensions of instrumental use: information relevance, information 
surplus, recommendation implementation and overall satisfaction. 

In this study, it was observed that credible design objectives obtain 
commitment. Requirement statements were tested, either formally or informally, 
for credibility. The credibility of the vital few requirement statements reinforces 
commitment to the design objective. Requirements without credibility are 
discounted, thereby reducing commitment to the stated design objectives. 


Evidence, inferences and propositions are provided below. 


"Post program approval then run in automatic; hard work but people already know what 
to do and want to do it.” - engineering manager 


Credibility ae Committment 


P15: As credibility increases commitment increases. 
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Objective Function — Substantive Development Accomplishment 
substantive development accomplishments are development actions 
which lead directly to real progress towards realizing the design objectives. 
Substantive development accomplishment, in the context of the concept 
development decision process, has two dimensions: concept commitment, and 
focused effort. Concept commitment represents the ability of a product concept 
to garner enough enthusiasm and support from the development team that it 
does not change during subsequent development activities. Focused effort 
describes a development process which is direct compared to one filled with 


detours. 


Concept Commitment 

Several authors claim commitment is a result of participation in the 

formulation stage of a project (Gupta, Raj & Wilemon 1986, Shapiro 1988, 
Moenaert & Souder 1990, Gupta & Wilemon 1990, Bailetti & Guild 1991). Gupta 
& Wilemon (1990) indicate that a lack of commitment is related to changes in 

product definition and low management support. 

In this study it was observed that design objective appreciation led to concept 
commitment which in turn led to reduced misdirected development effort. Concept 
commitment represents the level of support the product concept has earned from 
both the development team and their managers. Committed individuals ensure the 
necessary work get accomplished. Committed managers provide the necessary 
resources to support the team's efforts. Concepts with commitment don't change 
during the development process. Changing product concepts often make prior 
development activities useless creating waste and rework. 


Evidence, inferences and propositions are provided below. 


Z)k 


"We didn't have confidence, (therefore) we didn't want to tinker because it would be 
wasted effort.” - engineering manager 


Concept gp Misdirected 
Commitment O Effort 


P16: As concept commitment increases misdirected effort decreases. 


Misdirected Development Effort 

As mentioned previously, the absence of a clear product definition has 
been linked to instability in product and marketing plans (Gupta & Wilemon 
1990, Wilson 1990). Instability can manifest itself in significant changes in 
direction or in "creeping elegance" (Gupta and Wilemon 1990). Wilson (1990) 
found that product definition instability could lead to more staffing, funding and 
time. 

The System Dynamics literature has investigated the impact of project 
changes in a variety of development settings, e.g. construction (Homer et al. 199), 
research and development (Roberts 1964, 1978, Richardson & Pugh 1981), 
shipbuilding (Cooper 1980, Reichelt & Sterman 1990), and software (Abdel- 
Hamid & Madnick 1989, 1991), and consistently find project changes directly and 
indirectly increase development time and costs. Abdel-Hamid and Madnick 
(1989; p.1427) state: "the rework necessary to correct such software errors 
obviously diverts the project team's effort from making progress on new project 
tasks, and thus can have a significant impact on the projects progress rate." They 
explicitly model the relationship from progress-rate to forecast completion date 
to schedule pressure. Roberts’ discussion of research and development project 
control indicates that "there is no intrinsically correct measure either of 


engineering effectiveness or of problems solved or of the task left to be done....the 


92 


obvious concrete and measurable variables are often basically unrelated to the 
amount of effort required to get the job done" (1964; p.169). Roberts indicates 
(1964, 1978) that one result of this measurement difficulty is a delayed response 
to events which impact the development schedule. 

In this study, it was observed that a reduction in Misdirected 
Development Effort decreased development time. In addition to tangible time 
savings, reduced misdirected effort was observed to increase the sense of 
accomplishment of the team. It is assumed that the time saved through 
reductions in tangents, detours, missteps, etc. decreases Pressure for Progress 
and Labor-hour Requirements. 


Evidence, inferences and propositions are provided below. 


"This process would have provided a clearer vision, a straighter path to the end result...I 
see the process saving time by eliminating missteps" - design engineer 


Misdirected ‘ Development 
Effort S Time 


P17: A decrease in misdirected effort decreases development time. 
P18: A decrease in development time decreases pressure for progress. 


P19: A decrease in development time decreases labor-hour requirements. 


Constraints — Labor-hours 

Constraints are limits placed on decision variables. Although the list of 
conceivable constraints which can be imposed on the decision variables outlined 
above is considerable, one, required labor-hours, dominated the observations in 
this study of product concept development. As the innovation process is primarily 
informational (Moenaert & Souder 1990) it can be argued that mental capacity 


(labor) is the critical resource. The Labor-hour requirement gap represents the 
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difference between required laborhours and available laborhours. When Labor- 
hour requirements exceed availability the gap is large. In this study, labor 
availability was fixed by the team size. In every observed development team, the 
membership remainded the same or was reduced during the roughly six month 
observation period. The labor requirement can be driven by the project itself or by 
other projects team members participate in. In this study, it was observed that 
when a Labor-hour requirement gap exists there is an increase in concept 
development time. It was also observed that systematic concept development 
(comprehensive, collaborative, customer oriented analysis) increased the labor 
requirement. 


Evidence, inferences and propositions are provided below. 


"This process (CE) ts taking a long time; not just a long elapsed time because it is not 
calendar time. But in terms of people time it is extensive.” - marketing manager 


Labor 
or ————_® Requirement 
epth S 
Gap 


P20: As Analysis Depth increases labor-hour requirements increase. 


"We have sales quotas to meet, sales growth rates to meet, as a result the managers won't 
give time to gather and analyze the information (for products not yet defined).” - manager 


Labor Concept 
Requirement —————> Development 
Gap “ Time 


P21: As the labor requirement gap increases concept development time 
increases. 
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Integration 

Combining the relationships identified above into an integrated diagram 
allows us to better understand the interactions between the variables associated 
with the product concept development decision. Each arc of the diagram has 


been annotated with the relevant proposition number. 


Pressure 
for Progress P3 
P18 S S 4 
Delay 
= O S 
Analysi 8 i a1di 
— ted Goncer! ae ee ae 8. Prejudiced 
evelopmen Development P Sebi ce auc.) herspectives 


S ‘\ Time 
P19 S 
P21 
Bly Delay 
XQ S Labor-hour 9S P20 P7 
Requirement Be 2 


Misdirected Gap 
Efforts S S S O 
O o Process Contextual 
Nr prea LY Participation Awareness 
Concept 
Commit- P13 
tment 
S S P12 
No P11 
Credibility 
S 10 
P14 S 


Requirement 
Clarity 5 


This TIME vs. MARKET inductive system diagram can describe a vicious 
or virtuous cycle of product concept development depending on which decision 
variables are emphasized. A vicious cycle begins when pressure for progress 
leads to incomplete concept analysis and stakeholder prejudiced perspectives in 
decision analysis. The resulting decrease in functional integration further 


degrades analysis depth and reduces participation of all team members in the 
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concept decision process. The resulting lack of requirement clarity and 
credibility leads to low concept commitment and misdirected development 
effort. Ultimately, the waste and rework increases development time further 
increasing pressure for progress. 

The TIME vs. MARKET inductive system diagram can also describe a 
virtuous cycle in which an increase in customer orientation perspectives lead to 
an increase in Functional Integration and Contextual Awareness. As a result, a 
more thorough analysis, grounded in the context of the customer's environment, 
is conducted with the active participation of all development team members. 
This common appreciation of the concept decision process and outcomes leads to 
a higher degree or requirement clarity and credibility. In turn, commitment to 
the product concept is higher and misdirected development effort is reduced. 
Ultimately, development time is reduced, thus decreasing pressure for progress 
and labor requirements. 

The dynamics associated with either a TIME or MARKET emphasis in the 
expression time to market can be explained by the inductive system diagram 
above. The level of detail displayed in the diagram provides a more 
comprehensive causal map than currently exists in the literature. Additionally, it 
identifies the dysfunctional and unintended consequence which results from a 


TIME to market orientation during the product concept decision process. 


Plausible Rival Hypotheses 

The inferences and propositions integrated into the TIME vs. MARKET 
inductive system diagram above are the result of a comparative analysis of 
product concept development teams. The differences between the observed 
teams were both minimized and maximized, within the constraints imposed by 


company access. The full range of behavior observed in this study is accounted 
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for in this inductive system diagram. However, the actual implementation of the 
original research design does not preclude the elimination of some plausible rival 


hypotheses which will be discussed below. 


Senior Management Support 

Numerous authors describe the important role senior managers play in 
creating the Market Oriented organization (Shapiro 1988, Kohli & Jaworski 1990, 
Narver & Slater 1990). Gupta & Wilemon (1986, 1988, 1990) specifically identify 
senior management support as a necessary ingredient for creating interfunctional 
integration and cooperation. In this study, it might be argued that those teams 
using Concept Engineering received, or at least could be perceived as receiving, a 
higher level of support from their senior managers than the teams which did not 
use Concept Engineering. The original research design attempted to address this 
threat by using pairs of teams from the same division each of which received a 
beneficial treatment. Unfortunately, the actual design implementation precludes 


elimination of this threat. 


Decision Support Process 

White, Dittrich and Lang (1980) found that the more structured the 
interaction during group decision making, the greater the commitment to the 
group derived solution, as measured by the number of implementation attempts. 
Concept Engineering, as a complete decision support process (see chapter 1), is a 
structured decision making process. A plausible rival hypothesis to the TIME to 
MARKET dynamics presented above relates to the quality of the decision process 
employed by the MARKET oriented teams. 
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Janis (1985; p.167), based on studies of errors in strategic decision making, 
outlines seven major decision process criteria which are influence the quality of 
individual or group decisions. High quality group decisions: 

1) thoroughly canvass a wide range of alternatives; 

2) take account of the full range of objectives to be fulfilled; 

3) carefully weigh whatever is found out about negative and positive 
consequences that flow from each alternative; 

4) intensively search for new information relevant for alternative evaluation; 

5) conscientiously take account of any new information, even when the 
information does not support the course of action they initially prefer; 

6) re-examine the positive and negative consequences of all known alternatives 
before making a final choice; and 

7) make detailed provisions for implementing the chosen policy, with special 
attention to contingency plans. 

Janis (1985) states it is plausible to assume that failure to meet these criteria are 
symptoms of defective decision making that increase the chances of undesirable 
outcomes. Further, he states that the vigilance pattern of response is more likely 
than others to lead to decisions that meet the main criteria for sound decision 
making. The vigilant decision maker "searches painstakingly for relevant 
information, assimilates information in an unbiased manner, and appraises 
alternatives carefully before making a choice" (p. 184). From this perspective, 
vigilant decision makers, who succeeded in satisfying the above criteria for each 
stage (requirement identification, idea development and concept selection) of the 
concept decision process, will have more successful outcomes. 

The vigilant decision making argument would indicate that 
comprehensive analysis is a sufficient factor for success in the product concept 


decision process. Concept Engineering, as a complete decision support process 
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(see Chapter 1) satisfies Janis' requirements for high quality decisions as 


indicated in the table below. 


1) Thoroughly canvassesa__| In Transforming voices into Requirements (Step 
wide range of policy 4) 200-300 requirement statements are usually 
alternatives. developed. In Concept Generation (Stage 4) 
teams can generate over 300 solution ideas. 
2) Takes account of the full | In Concept Screening (Stage 5) all concepts are 
range of objectives to be screened against the set of key customer 
fulfilled. requirements. 
3) Carefully weighs whatever | In Concept Selection (Step 14) final concepts are 
is found out about negative | screened not only against customer 

and positive consequences requirements (positive consequences) but also 
that flow from each against organizational, environmental and 
alternative. technological constraints (negative 
consequences). 
In Collecting the Voice of the Customer (Step 2) 
and Developing and Administering 
Questionnaires (Step 7) active information 
discovery and collection occurs. 
Transforming Voices into Requirements (Step 4) 
requires every customer statement be developed 
into a customer requirement for possible 
inclusion and development in the product 
concept. 












































































4) Intensively searches for 
new information relevant for 
alternative evaluation. 


5) Conscientiously takes 
account of any new 
information, even when the 
information does not support 
the course of action they 
initially prefer. 
6) Re-examine the positive 
and negative consequences 
of all known alternatives 
before making a final choice. 
7) Make detailed provisions 
for implementing the chosen 
policy, with special attention 
to contingency plans. 












Solution Screening (Step 13) and Concept 
Selection (Step 14) require all developed 
concepts to be evaluated in a matrix against 
customer / environmental requirements. 
This activity is not formally part of Concept 
Engineering. However, detailed design 
specification activities usually occur subsequent 
to concept approval. 






















In this study, those teams which were successful in developing product 


concepts achieving a high degree of commitment from development team 
members and managers were also those teams which completed a 


comprehensive analysis (Concept Engineering) which satisfied the requirements 
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outlined by Janis. However, one team which used Concept Engineering was not 
successful in developing concept commitment. This team had a relative 
emphasis on TIME versus MARKET and was under considerable pressure for 
progress. Although a relatively complete investigation was conducted not all 
participants were active in the entire concept decision process, e.g. three 
members did not conduct customer interviews, two members did not participate 
in idea generation. As a result, acommon appreciation of the design objectives 
was not obtained and commitment to the product concept was low. This would 
indicate that the decision process criteria outlined by Janis may be necessary but 


are not sufficient for success in the product concept development process. 


Market Orientation Contingency 

When is too much of a good thing not a good thing? Several authors 
specify contingencies with respect to the effectiveness of market orientation 
(Houston 1986, Gupta & Wilemon 1986, Souder 1988, Kohli & Jaworski 1990, 
Moenaert & Souder 1990, Narver & Slater 1990). Souder (1988) created a matrix 
with customer sophistication on one axis and R&D sophistication on the other 
axis and prescribes different tactics for the twelve resulting cells. Kohli and 
Jaworski (1990) propose that when the general economy is weak there is a 
stronger relationship between market orientation and business performance. 
Gupta and Wilemon (1986) propose that if a firm values being first in the market 
with new products it will benefit from higher integration. Kohli and Jaworski 
(1990) and Moenaert and Souder (1990) propose that market orientation is less 
valuable in environments with technological uncertainty but more valuable in 
environments with consumer or competitor uncertainty. Several authors address 


the issue that the cost of obtaining more information at some point exceeds the 
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value of the benefit to be derived from knowledge gained. (Houston 1986, 
Moenaert & Souder 1990, Narver & Slater 1990). 

Most of the above contingencies address when more market orientation is 
beneficial. However, in the case where technological uncertainty reduces the 
value for market orientation, the basic dynamics outlined in the high level 
inductive system diagram still appear to provide valuable insight, i.e. pressure 
for progress will reduce systematic analysis reducing evidence, credibility and 


substantive accomplishments. 


Conclusion 

This investigation shows that a relative emphasis on TIME creates an 
environment where pressure for progress encourages development teams to 
conduct incomplete analysis oriented to self-interested outcomes during the 
concept development decision process. The resulting lack of design objective 
appreciation and commitment by the development team leads to waste and 
rework in subsequent product development activities. On the other hand, a 
relative emphasis on MARKET can increase design objective appreciation, 
credibility and commitment but increases the time required in concept 
development. These dynamics indicate that increased time spent systematically 
developing a product concept, which remains stable over the balance of the 
development process, results in getting a product to market faster. Schmenner 
(1988) reminds us of the applicability of the fable of the tortoise and the hare to 
product development acceleration. The tortoise won the race with a diligent, 
focused effort and the hare, while very fast, had a pattern of stops and starts in 


his detour-filled route to losing the race. 
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Chapter 5: Time-to-MARKET Management 
Diagnosis 


The dynamics of TIME versus MARKET orientation activities has 
important implications for the management of product concept development 
activities. The emphasis on TIME clearly implies schedule related measurement 
and monitoring. The measurement and monitoring requirements associated 
with an emphasis on MARKET in the expression Time to Market is not so clear. 
This chapter will explore methods! managers can use to diagnose concept 


development activities in a time-to-MARKET oriented environment. 


Product Concept Decision Process 

The product concept process can be described to follow the general 
decision process structure outlined by Mintzberg and colleagues (1976) in their 
study of unstructured decision processes. In the context of concept development, 
the three general phases of the decision process are: Requirement Identification, 
Idea Development and Concept Selection. All three phases — identification, 
development and selection — were observed, to a greater or lesser extent, in each 
development team in this study. 

In the identification stage, both recognition and diagnosis routines were 
used to clarify and define the requirements which would drive the product 
concept. In the development phase, search and design routines were employed 
to generate ideas which constitute potential solutions to requirements. Finally, in 
the selection stage, screening, evaluation and authorization routines were 
employed either by the development team, or a management team, to select and 


authorize a product concept for additional investment. Janis (1985) indicates that 


! Metrics presented in this chapter are derived from work conducted with the CQM Research 
Subcommittee on Product Development Metrics (CQM 1992). 
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the effectiveness with which these individual activities are carried out influences 
the outcome of the decision process — the product concept which determines 
75% of the product's lifecycle costs (NRC 1991). 

Janis (1985) outlines seven major decision process criteria which influence 
the quality of individual or group decisions. The table below identifies where in 


the product concept decision process these criterion are most relevant. 


Concept Selection 
- screening 

- recognition - evaluation 

- diagnosi 1p -authorization 


1. Canvasses a wide range of 
alternatives 

2. Takes account of full range 
of objectives 

3. Carefully weights 


pros/cons of alternatives 

4. Intensively searches for new 
relevant information 

5. Conscientiously takes 
account of new information 

6. Re-examines pros/cons 
prior to making choice 

7. Make detailed 
implementation plans 





To identify a comprehensive set of customer requirements an intensive search of 
market segment information is required and this new information must be 
incorporated into concept deliberations. To develop an extensive set of potential 
product concept ideas, deliberate exploration must occur around the set of key 
customer requirements. Finally, to determine the best product concept requires a 
systematic comparison of each candidate concept's ability to satisfy key customer 


and organizational requirements. 
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This mapping of decision effectiveness criteria onto the product concept 
decision process identifies high leverage management diagnosis opportunities. 
Specifically, within each phase (identification, development, selection) of the 
product concept decision process, the relevant decision criterion are aligned with 
variables associated with time-to-MARKET orientation. These variables are then 
operationalized in a manner which is practical for use in diagnosing actual 


product concept development activities. 


Requirement Identification 

Mintzberg et al. (1976) observed that the identification phase consists of 
both recognition and diagnosis activities. They defined diagnosis as "the tapping 
of existing channels and the opening of new ones to clarify and define the issues" 
(1976; p.254). Three decision process criteria are proposed to have potential 
relevance in requirement identification activities: the search for new information, 
the range of alternative generation, and the conscientious consideration of new 
information. Based on the research associated with this study, Customer 
Orientation, Crossfunctional Collaboration, and Credibility are proposed as the 


variables with the most direct impact on the relevant decision process criteria. 


Intensive Range of Conscientious 
search for new | alternative idea | consideration 
information generation of new info. 


a een CTC 
Credibility ee ee ae 


Customer Orientation represents a willingness to search beyond 









traditional organizational perspectives. Customer Orientation results in 
developing an understanding not only of the stated customer requirements but 


also of the factors which influence those requirements (Kohli & Jaworski 1990). 
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Customer Orientation leads to a more thorough investigation of the customer's 
use environment. 

In Crossfunctional Collaboration, different functional groups work 
together to form a synthesis of their respective perspectives. This joint analysis 
of the opportunities leverages the strengths of each functional group in the 
creation of new insight (Shapiro 1988, Gupta & Wilemon 1986). Crossfunctional 
Collaboration, compared to Partisanship, leads to the development of a wider 
range of customer requirements. 

Credibility is a function of information credibility and source credibility 
(Gupta & Wilemon 1988). In this study, it was observed that Credibility was a 
function of Contextual Awareness and Process Participation. Contextual 
Awareness caused development team members to develop empathy for the 
customer which increased the (information) credibility of customer requirements. 
Similarly, Process Participation allowed team members to personally participate 
in the creation of the new information increasing its (source) credibility and 
subsequent utilization. Credibility is required for conscientious consideration of 
new information. 

The Customer Visitation Matrix and a Process Participation Matrix can be 
used by managers to assess the opportunities for developing customer 
orientation, crossfunctional collaboration, process participation and contextual 


awareness. 


Customer Visitation Matrix 
The Customer Visitation Matrix (Burchill et al. 1992) can assess the degree 
to which the development team is exploring potential market and the 


opportunities individual team members have to develop contextual awareness. 
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Across the top of the matrix, list important customer types, e.g., Lead Users, 
Demanding, Former, Satisfied, etc. Down the side of the matrix list important 
market segments and the specific companies to be visited during market 
exploration. In the intersecting cells, write down the name(s) of the development 
team members who conducted the customer visit. In the hypothetical example 
below, the development effort focused on only New England and Mid-Atlantic 
market segments. Additionally, Smith participated in all customer visits while 
Green only visited retailers and not end-users. Ideally, to develop contextual 


awareness participants would visit a representative cross section of customer 





types. 
Customer aaa 
Selection / he Demanding Lost | Retailers | Total 
Vistation zea 
New Brown & | Smith & Smith & Green & 7 
England Smith-2 | Jones-2 | Jones-1 | Smith-2 
Middle Brown & Smith & Green & 
Atlantic Smith - 2 Jones-1 Smith-1 
South 
Eastern 


Process Participation Matrix 

The Process Participation Matrix can assess the degree to which individual 
members of the development team participated in activities associated with 
concept development. Across the top of the matrix place the departments 


involved in concept development activities. Down the side of the matrix indicate 


2 See Step 1 of the Concept Engineering Manual (Appendix A) for additional description. 
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the stages and steps associated with concept development. In the intersecting 
cells, write down the names and hours of the development team members who 
participated in the various activities. (This process can be automated through the 
use of appropriate job numbers in the labor accounting system.) In the 
hypothetical example below, the total time investment is fairly equal in each 
department. However, the marketing department, Smith in particular, did the 
majority of data collection and requirement generation, while the engineering 
department, White, who did not make customer visits, did most of the 
requirement evaluation. This imbalance in Process Participation and Contextual 


Awareness could adversely impact Requirement Clarity and Credibility. 


Dept. M.H, 


Enginee 
Person 


Collection 


Requirement 
Generation 


Requirement 
Evaluation 





Idea Development 

The Development Phase observed by Mintzberg et al. (1976) consists of 
both search and design routines. They indicate four different kinds of search 
behavior: memory, passive, trap and active and two types of design activities 


that result in either custom-made or modified solutions. Three decision process 
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criteria are proposed to have relevance in idea development activities: the search 
for new information, the range of alternative idea generation and accounts for the 
full range of objectives. Based on research associated with this study, Customer 
Orientation, Crossfunctional Collaboration, and Requirement Clarity are 


proposed as the variables with the most impact on the relevant decision criteria. 


Intensive search Range of Accounts for 
for new alternative idea| full range of 
information Peneration objectives 


(CastomerOrentation [| Xf 
Crossfuncional Collaboration | S| XT 
ee x 


Requirement Clarity 






















Customer vs. Stakeholder Orientation represents a willingness to search 
beyond traditional organizational perspectives. Rothwell et al. (1974) conclude it is 
desirable “whenever possible" for designers to visit customers to study their 
technical requirements and also the actual operating conditions. Bailetti and Guild 
(1991; p.91-92) cite three reasons for designers to visit customers: 1) added insight 
and useful knowledge, 2) acquiring knowledge depth that facilitates technical 
activities, and 3) increasing designer acceptance of the results. Customer 
Orientation should produce additional information on technical considerations. 

In Crossfunctional Collaboration, different functional groups work together 
to form a synthesis of their respective perspectives. Different functional groups have 
different frames of reference and skills which they bring to focus on aspects of the 
technology-market environment (Van de Ven 1986, Dougherty 1992). These diverse 
perspectives, if integrated, should generate a wider range of alternatives compared 
to those generated from partisan perspectives. Requirement Clarity results from 
understanding the vital few requirements and the relative priorities within this set of 
requirements. Assessing the degree to which design objectives are met first requires 


a clear understanding of the design objectives. Requirement Clarity is a necessary 
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condition for determining if the developed ideas account for the full range of 
objectives. 

The use of the Customer Selection/Visitation Matrix and a Process 
Participation Matrix, reflecting appropriate stages and steps, can provide some 
diagnosis capability of the idea development process. Additionally, the Idea Count 
Chart and the Requirement Utility Matrix can also be used by managers to monitor 


idea generation activities. 


Idea Count Chart 

The Idea Count Chart can assess the range of alternative ideas generated. 
Concept development activities can be decomposed? in multiple ways, e.g. by 
customer requirement, functional requirement, etc. The Idea Count Chart displays 
the number of ideas generated in each decomposition category. In the example 
below, a customer requirement decomposition was used and the number of unique 
ideas associated with each requirement were counted. In this hypothetical example, 
only one unique idea was generated to address requirement number 5; this may 


represent an area requiring additional idea generation. 


Number of Unique 
Ideas Generated 
- NO PB UW Oo 





a 
ee ee Ep eee eee se ee Be Be eee ne pe ieee eb net epee Mie ses 5 8 Bo Be eb eee es Mae be be bt be eee ee ee ete at et ate pees 


cr: Cr*2 .Ch* 5. Ch= 4eGra cr*26 





Customer Requirement Number 


3 See Step 10 of the Concept Engineering manual (Appendix A) for additional description. 
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Requirement Utility Matrix 

The Requirement Utility Matrix can be used to assess the relative clarity 
and flexibility of requirement statements. It was observed that the customer's 
articulation of their need is often ambiguous, does not clearly state the 
requirement and leaves a great deal of flexibility for designer interpretation. On 
the other hand, it was observed that the designer's articulation of a customer's 
need was Often a very specific solution; thus greatly restricting designer 
flexibility. Unfortunately, specifying a requirement as a specific solution, while 
very clear, restricts the number of generated ideas to the one solution. To 
develop the matrix, each requirement statement is reviewed and its relative 
clarity and flexibility are assessed and the requirement is plotted in the 
appropriate cell. Requirements in the upper left hand corner of the matrix tend 
to be written as solution statements not requirement statements. Ideally, 
requirement statements have both high clarity and flexibility and would be 


located in the upper right hand of the matrix. 


Example from the Stripping Basket 


CR 1: The basket has a velcro fastener. 
CR 2: The basket is releasible with one hand . 
CR 3: The basket is durable. 
CR 4: The basket is simple. 





LOW Relative Requirement Flexibility High 
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Concept Selection 

The Mintzberg study (1976) identified three routines in the selection phase: 
screen, evaluation-choice, and authorization. Three decision process criteria are 
proposed to have potential relevance in concept selection activities: accounts for 
full range of objectives, carefully weights pros/cons and examination of pros/cons 
prior to choice. Based on research associated with this study, Process 
Participation, Requirement Clarity, and Traceability are proposed as the variables 


with the most direct impact on the relevant decision process criteria. 


a Accounts for Carefully Examines 
full range of weights pros/cons prior 
objectives oros/cons to choice 

PRequirement Clarity 


Traceability 












Process Participation represents the active involvement of participants in 
the complete decision process which includes requirement identification and 
idea development, in addition to concept selection. Inclusion in the decision 
process enables participants to understand how their function relates to other 
functions and also their function relates to the overall innovation (Van de Ven 
1986; p.600). Process Participation enables development team members to 
incorporate a more complete range of objectives in their deliberations. 

Requirement Clarity, by definition, includes the key requirements and 
their relative priorities. Without these conditions it would not be possible to 
evaluate pros and cons of concept alternatives. 

Traceability of the decision process and outcomes was important for 
justifying the decision choices. In this study it was observed that the rationale for 


decision choices made early in the concept development process was required 


112 


during final concept evaluation. Traceability provides the vehicle for ensuring 
these factors are available for consideration during concept selection. 

In addition to the use of the Process Participation Matrix, reflecting 
appropriate stages and steps, the Concept Selection Matrix can be used to 


diagnose final concept selection. 


Concept Selection Matrix 

The Concept Selection Matrix (Burchill et al. 1992) can be used to assess 
relative pros and cons of the generated (selected) concepts against the full range 
of objectives. Across the top of the matrix list the concept alternatives. Down the 
side of the matrix list the design objectives and constraints (organizational, 
technical, environmental, etc.). Select one concept to be the reference datum and 
evaluate the strengths and weaknesses all other concepts relative to the datum. 
In the example below, "1" is much worse than the datum, "3" is the same as the 
datum, and "5" is much better than the datum. In this example, concepts #3 and 
#8 are comparable with respect to satisfying customer requirements but concept 


#8 requires less risk and resources and appears to dominate. 
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Conclusion 

The emphasis on TIME in the expression Time to Market clearly implies 
schedule related measurement and monitoring. However, the measurement and 
monitoring requirements associated with an emphasis on MARKET in the 
expression Time to Market is not so clear. In this chapter, I identify high 
leverage opportunities for management attention in the concept development 
process and propose relevant diagnostic devices. These measurement and 
monitoring devices are designed to assist management of the product concept 


decision process in time-to-MARKET oriented environments. 
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Chapter 6: Next Steps and Concluding Comments 


This thesis presents Concept Engineering as a complete decision support 
process for enhancing product concept development, explores the dynamics of 
TIME vs. MARKET orientation in product concept development, and introduces 
Inductive System Diagrams as a variable development and integration 
enhancement for substantive theory generation. In this chapter, I will outline 


some potential next steps for continuing research in each of these areas. 


Concept Engineering 

Concept Engineering is currently being applied to approximately two 
dozen concept development efforts in ten organizations within the Center for 
Quality Management. The expanded application base includes small 
entrepreneurial concerns developing follow-on products, a Fortune 50 company 
on a large scale development effort, and a state government agency in the 
development of a new law. This application base provides a rich comparative 
setting to explore two broad investigative themes — Concept Engineering 


application contingencies and Concept Engineering process improvements. 


Concept Engineering Process Improvements 

There are two general process improvement themes related to Concept 
Engineering which can be explored. The first is improvement to the overall 
process, specifically focused on reducing the decision time. The second area of 
opportunity involves investigating enhancements to particular decision aids, 


specifically the requirement transformation process and Kano's analysis. 
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Concept Engineering Acceleration 

The actual time to complete the Concept Engineering efforts in this study 
was substantially longer than the projected time. Applying the framework of 
Millson et al. (1992), it should be possible to identify potential areas for 
simplification, step and delay elimination, and parallel processing. The 
traditional roadblock for applying these acceleration strategies is the requirement 
for a thorough understanding of the targeted process. Fortunately, due to the 
combined experience of the User's Group, a fairly high level of Concept 
Engineering process knowledge exists. Additionally, the spirit of mutual 
learning, which is a cornerstone of the CQM, enables faster turns of the Plan-Do- 
Check-Act cycle through sharing of experiences across organizations. Therefore, 
within this environment, research to accelerate the development of market 


orientation within development teams would be desirable and possible. 


Concept Engineering Decision Aid Improvements 

A consistent observation during this study was the tendency for 
"requirement" statements to represent specific solutions rather than be a 
reflection of customer needs. The benefit of writing a requirement statement as a 
specific solution is reduced ambiguity. Unfortunately, the expression of a 
requirement as a functional solution severely restricts the designer's flexibility to 
make tradeoffs. It should be possible, using Likert-type scales for example, to 
assess the relative clarity and flexibility of requirement statements. These 
assessment could be made either by the team members themselves or by expert 
judges, either within or outside a organization. These data can be analyzed to 
assess their impact on development efficiency, effectiveness and innovation. The 


results could conceivably have a significant effect on product definition practices. 
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Kano's analysis (Kano 1984) helps us understand the relationship between 
the fulfillment (or non-fulfillment) of a requirement and the satisfaction (or 
dissatisfaction) experienced by the customer. Extrapolating Herzberg's (1966) 
motivator-hygiene theory to product quality characteristics Kano discovered that 
the relationship between fulfillment of a need and the satisfaction or 
dissatisfaction experienced is not necessarily linear but can be separated into four 
main categories: attractive, one-dimensional, must-be and indifferent. For 
example, when an attractive item goes unfulfilled the result is not dissatisfaction 
but the absence of satisfaction; in contrast, fulfilling a must-be item does not 
produce satisfaction. Currently, it has been observed that the construction of the 
questions and response scales associated with Kano's analysis is problematic for 
some development professionals. Additionally, Kano's analysis has not been 
systematically evaluated (or at least published in English language academic 
journals) in the context of traditional marketing research techniques. This is an 
area of investigation with potentially significant implications for product 
development resource allocation decisions. Preliminary work is underway for a 
collaborative effort between myself and Duncan Simester (1993 MIT-Marketing 


Ph.D. candidate) to conduct this more systematic analysis. 


Concept Engineering Application Contingencies 

With approximately two dozen Concept Engineering applications 
underway, covering the spectrum of market-technology uncertainty, a more 
thorough understanding of the contingencies associated with applying Concept 
Engineering can be developed. Current applications range from simple product 
line extensions, to radically new product concepts, to the development of new 


state laws. This environment can provide a rich comparative setting for the 
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development of contingency theories related to the product concept decision 
process. 

All of the completed Concept Engineering applications have been product 
extensions for existing markets or existing technologies. However, some 
members of the User's Group believe Concept Engineering is best suited for 
developing products for new markets or technologies. One effort currently 
underway is pursuing a completely new product concept which they hope will 
create entirely new markets. Another effort is exploring potential market 
applications for new technology. These extremes, anchored by base cases in 
market-technology extensions leads to a potentially productive line of 
comparative research addressing the contingencies of Concept Engineering in 
particular and Market Orientation in general. 

The effort to develop a new state law represents a service application in 
which constituencies have clearly conflicting positions. Prior Concept 
Engineering efforts have investigated diverse market segments, i.e. North 
America, Europe and Asia. However, it was assumed in those efforts, that if a 
clear conflict developed, multiple products would be developed or the decision 
choice would be made on the basis of the largest profit potential. In the case of 
the state law, it is not possible to develop multiple versions and it is not obvious 
how "profit potential" would be assessed. However, based on the work of the 
Harvard Negotiation Project (Fisher & Ury 1981), a dominant strategy for 
successful negotiations is to focus on the common interests instead of conflicting 
positions. In the state case, it has been possible to identify and articulate the 
common requirements of all constituencies in addition to their requirement 
differences. This case represents a potentially useful process application of 


Concept Engineering which has not been previously explored. 
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Time-to-Market Dynamics 

Chapter Four of this thesis presented numerous propositions related to 
Time-to-Market dynamics which need to be validated. Some of the constructs 
identified as influential have been operationalized and investigated in other 
studies (). Other constructs (e.g. traceability, process participation, contextual 
awareness) have not been investigated and their significance on product concept 
development has not been statistically validated. Chapter 5 attempts to 
operationalize some of these variables from a managerial perspective which may 
be insufficient for a study attempting to apply traditional academic standards of 
Statistical significance. Additionally, even after the hurdle of developing 
multiple construct operationalizations is overcome, the data collection process 
will be formidable. 

My experience, at every company in this study, indicates that product 
concept development data is not systematically collected, if it is collected at all. 
Concept development activities are traditionally ad hoc and corporate product 
development data collection systems typically begin after concept approval not 
before. However, sufficient data may be available to assess the basic proposition 
that credible design objectives result in stable product concepts and the resulting 
reduction in misdirected effort reduces total development time. 

Many companies use a product development phase review process with a 
formal "Concept Approval" stage. Conceivably at this point, a product definition 
exists which can be assessed for clarity and credibility, e.g. using Likert-type 
scales. Concept stability can be determined by reviewing engineering change 
notices assuming the records exist and it is possible to distinguish those 
engineering changes related to concept instability from those related to other 
sources, i.e. manufacturing requirements. This information can be compared to 


actual development time versus the original development schedule. This final 
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comparison is also dependent upon several assumptions regarding schedule 
forecasting reliability and project characteristics. However, a test for collecting 
the data described above, with the assistance of the company's Product 
Development Quality Assurance Department, was conducted in one company 
from this study. This limited data collection effort was difficult but does indicate 
the feasibility of investigating some of the Time-to-Market propositions utilizing 
empirical data. 

Another path to model validation could be through system dynamic 
simulation. In the system dynamic approach, model validation follows from a 
multi-method analysis of computer simulations (Forrester & Senge 1980, Maas & 
Senge 1980, Richardson & Pugh 1981, Sterman 1984, Barlas 1989, Barlas & 
Carpenter 1990). System dynamicists have a long history of successful 
simulation analysis of development project management (see for example: 
Roberts 1964, Richardson & Pugh 1981, Abdel-Hamid & Madnick 1991, Sterman 
1992). This previous work, assesses the impact of project changes on 
downstream development activities given initial tasks. However, as the existing 
work does not model the early concept development activities addressed in this 
research an opportunity exists for this research to extend previous dynamic 
models of project management. 

The Time vs. Market inductive system diagram would serve as the 
conceptual model around which a formal computerized model can be built. 
Model formulation is the process of transforming the conceptual model into 
equations which increase the precision with which the system structure is 
specified. This precision, which is a necessary condition for conducting the 
computer simulation and analysis, also reduces the ambiguity of the causal-loop 
diagram (Richardson and Pugh 1981). For example, in this research it is 


proposed that reductions in Development Time would reduce Pressure for 
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Progress (Chapter 4, Proposition 18). Formulating this relationship might 
produce the equations below: 


Pressure for Progress = 
1 + (Expected Development Time - Scheduled Development Time) / Scheduled Development Time 


Expected Development Time = Perceived Work Remaining / Perceived Work Rate 
Perceived Work Remaining = Scheduled Work - Work Completed + Recognized Rework 


Perceived Work Rate = (Work Completed - Recognized Rework) / Labor-hours Invested 


These four equations clearly demonstrate how the formulation process 
fleshes out the skeletal structure of the inductive system diagram by specifying 
the system substructure of levels and rates. Formalizing the detailed dynamics 
of the entire Time vs. Market inductive system diagram will allow for a more 
precise definition of the relationships and subsequent simulation analysis of the 


propositions presented in this research. 


Inductive System Diagrams 

Two research themes could be pursued directly from the initial work on 
Inductive System Diagrams. First, a more extensive reliability assessment can be 
conducted and second, research related to enhancing the power of the diagrams 
should be pursued. 

The reliability assessment of Inductive System Diagrams needs to be 
conducted on a larger sample of testers, some of whom are experienced 
qualitative researchers. Additionally, the assessment of the diagrams should be 
conducted by a panel of trained evaluators rather than a single person to increase 
result reliability. Finally, given larger sample sizes statistical analysis of the 
results can be conducted. 

The rate-to-level limitations of causal-loop diagrams has been addressed 


by several systems dynamists through the use of flow diagrams (Forrester 1971, 
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Goodman 1974, Richardson 1981) and Policy Structure diagrams (Morecroft 
1982). Prior attempts at representing this additional structural detail 
unfortunately make the schematic much more difficult to comprehend by the 
uninitiated. However, it should be possible for the analyst to employ these 
structural insights in the development and description of their models even if the 
detail is absent from the presentation schematics. 

Additionally, Inductive System Diagrams can be extended by 
incorporating reference mode analysis into the development process. Reference 
modes clearly specify the dynamic behavior of interest in the system under 
investigation (Randers 1980, Richardson and Pugh 1981). Usually reference 
modes are based on actual historical data but they can also be created from 
expert assessments (Randers 1980, Richardson and Pugh 1981). Reference modes 
can be described either graphically or verbally but they must indicate the 
appropriate time dimensions of the variables described (Randers 1980, 
Richardson and Pugh 1981). Reference modes can help identify which variables 
should appear in the model (Randers 1980). Therefore, reference mode analysis 
could assist not only in the development of variables through theoretical 
sampling and coding but also in the elimination of variables during diagram 


integration. 


Concluding Comments 

At the highest level of abstraction, the basic lessons learned from the analysis of 
the Time-to-Market dynamics we were taught long ago: "Do it right the first time; 
because if you don't make time to do it right, you'll have to make time to do it over." 
One major contributing factor to the dysfunctional, unintended consequence of 
product concept development acceleration is the lack of product concept decision 


process understanding. Concept Engineering, as a complete decision support process, 
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clearly defines the steps and transitions associated with the product concept decision 
process and therefore has the potential for accelerating the development of market 
Orientation within product development teams. 

The development of Concept Engineering, the Time-to-Market analysis and 
Inductive System Diagrams all result from "cross-functional" collaboration. Without a 
common commitment to mutual learning between the initial companies and 
researchers at MIT, Concept Engineering would not have evolved into a complete 
decision support process. Without the companies granting the researcher extensive 
access to their product development activities, participants and managers, the rich 
comparative setting, essential for generating substantive grounded theory, would not 
have developed. Without the inter-disciplinary involvement of Operations 
Management faculty and Behavioral Studies faculty, the dialogue which resulted in 
Inductive System Diagrams would not have developed. Finally, without an overall 
commitment, by the thesis committee, to a partnership between industry and academia 


in the research of Total Quality Management, this thesis would not have developed. 
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Preface 


This manual is designed to assist organizations in focusing on their 
customers’ requirements in developing design concepts for products or 
services. Concept Engineering is a process for determining the customer's 
key requirements, creating a measurement strategy for assessing 
compliance with the requirements, and developing a strong product 
concept that satisfies the requirements. 


Document! History 


The Concept Engineering method and manual have been developed in 
the true spirit of mutual learning and collaboration between member 
companies of the Center for Quality Management (CQM) and MIT. The 
material presented in this manual is the result of many Plan-Do- 
Check-Act improvement cycles applied to both the teaching and use of 
this method. 


e The foundation for Concept Engineering began with a series of 
lectures by Professor Shoji Shiba at MIT and the COM in the fall of 
1990. 


e The first outline of the process was developed by Gary Burchill 
(USN/MIT) in the winter of 1990. 


e The first outline of the manual was developed by Gary Burchill 
(USN/MIT), Diane Shen (BBN), Ron Santella (GenRad), and Rich 
Lynch (Analog Devices) in the spring of 1991. 


e The first version of the manual (CQM 7P),written by Gary Burchill 
(USN/MIT), Diane Shen (BBN), and Ron Santella (GenRad), was 
published in November 1991. 


e The second version of the manual (CQM 71), written by Gary 
Burchill (USN /MIT), Diane Shen (BBN), Erik Anderson (Bose), 
David Boger (Bose), Chris Bolster (MIT), and Bill Fetterman 
(Analog Devices), with editing assistance from Kenny Likis (BBN) 
and Deborah Melone (BBN) was published in September 1992. 


e¢ The authors would like to thank the development teams in BBN, 
Bose, Analog Devices, and Polaroid for their feedback on Concept 
Engineering and Dave Walden (BBN) for his encouragement and 
critique. 
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Critical Assumptions 


Two critical assumptions have been made in the preparation of this 
manual: first, that the users of this manual are familiar with and 
competent in the KJ method; second, that the scope (minor product line 
extension, major new product introduction, etc.) of the development 
project has been at least preliminarily defined before entering step one, 
Exploration Planning. 


If the first assumption does not hold, we recommend that KJ skills be 
obtained (through COM courses) to maximize the potential benefit of 
this method. If the second assumption does not hold, i.e., the scope of 
the development project is very broad or vague, several iterations of 
the early steps of this method may be required before a market 
opportunity is identified. 


The example used throughout this 
manual 


This manual is designed to provide users with a quick introduction to 
the underlying concepts and methodology for each step. Steps are 
supported with examples, tips, and worksheets where appropriate. 
The appendix includes a glossary and further detail about selected 
concepts. 


All examples come from a case study provided by Gary Burchill from a 
project on which he worked at MIT. The product developed by the MIT 
team was a saltwater flyfishing stripping basket. 


A stripping basket is a device used by saltwater fly fishermen to collect 
their line before they cast out the line. Typically it is a store-bought or 
home-constructed plastic container with four sides and a bottom, which 
is strapped to the waist or chest of the fisherman. Before casting, the 
fisherman lays the fishing line into the container so the line will play 
out easily when the cast is made. The process of retrieving the line and 
placing the line in the container is called “stripping.” 


The goal of Gary Burchill and his MIT colleagues was to design a better 
stripping basket. Their basket was praised by the Outdoors Editor of 
the New York Times in his feature on Sunday, September 1, 1991. First 
year sales of their product has been ten times that of the product it 
replaced. 


Other examples of the application of Concept Engineering are 
available from various CQM companies. 
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Introduction 


Why Bother? 


A recent study conducted by the National Research Council states that 
over 50% of a product's life-cycle costs are determined in the concept 
formulation phase of product development, and that approximately 
75% of life-cycle costs are committed by the end of concept validation. 
Yet many companies spend little effort in these phases of product 
development and do not have effective methods for developing product 
concepts which they are confident will satisfy their customers. Unless 
people have an effective method for understanding the customers’ needs 
and finding a product concept to meet them, companies will be locked 
into unsatisfactory concepts which drive the life-cycle costs of their 
products. Concept Engineering is a process designed to provide such a 
method. 


Life Cycle Cost Commitment 


Lie Cycle Phases 


1. Define vse pattems 
2. Define alternatives 
3. Develop attematives 
4. Freeze sisystoms 
5. Prove teasbdiity 
6. Provide preliminary designs 
7. Provide detailed drawings 
| 8. Provide manufacturing plans 
9. Product 


“J 
On 


Cummulative percentage of 
life cycle costs affected 
& 


N 
On 





Full Scale Use 
Development 


Concept 
Formulation i 
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Production 
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The National Research Council study outlines the troubled state of US 
engineering design skills. Interviews with senior managers associated 
with product development at CQM member companies reinforce the 
findings of the National Research Council report. These senior 
managers were asked several questions, one of which was to describe 
images that come to mind when they think about their product 
development process. These observations, captured by the KJ shown on 
the following page, emphasize the importance of establishing a 
concept engineering process. 


Market-In Model 


Concept Engineering is built on two models introduced to the CQM by 
Professor Shiba: Market-In and WV problem solving. 


The "Market-In” attitude expands the horizon of how people and 
organizations view their job responsibilities. The output of one's effort 
is not the end objective of work, but instead the means by which to 
satisfy the customer. The end objective is customer satisfaction. 


"Market-In" 


Satisfaction of Customer 
(outside and inside) 





In contrast, the product-out attitude defines its task as first designing 
and building the product or service, and then convincing customers that 
the product or service really meets their needs. 
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WV Model 


The WV model,Professor Shiba’s extension of Professor Kawakita’s W 
model, is usefulfor framing the product development process. It starts 
with a broad-based exploration of essentials and, by alternating 
between the level of thought (ideas and concepts) and experience 
(data), key issues are progressively defined in ever finer detail. 


7. Reflect on Process 6. Standardize 





| Sense Problem | Problem L'-segnaraee 4 4. Plan Solution and... Solution and .. 
Level of conse Problem | 
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Thought Eee ee 
Formulate |] 2. Collect 3. Causal |} Implement | | 5. Evaluate 
P Problem Data Analysis Solution Effects 

Level of 
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Le Process Control 
Reactive Improvement 
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Concept Engineering 


Experience saseumaxmnantm 





Concept Engineering is a customer-centered process for clarifying the 
"fuzzy front end” of the product development processthat comes before 
detailed design and implementation. It is a conceptual model, with 
supporting methodology, for developing product concepts. The process 
alternates between the level of thought and level of experience in a 
way that allows participants to understand what is important to 
customers, why it is important, and how it will be measured and 
addressed. It is a customer-centered process of data collection and 
reflection designed to develop product concepts that will meet and 
exceed customer expectations. 
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Concept Engineering Stages 


Develop Understanding of 


Customer's Environment 





Convert Understanding intc 
Customer Requirements 


Operationally Define 
Requirements for 
Downstream Development 






Generate Concepts 


Select Concepts 
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Stage 1: Understanding the Customer's 
Environment 


In stage 1, an exploration plan is developed based on the project scope, 
which identifies the customers to be visited and the information, 
broadly defined, which is being sought. The customer visits are 
conducted with an emphasis on collecting notes on verbatim customer 
statements and field observations. Then, the development team 
develops a mental model of the customer's environment to create a 
contextual anchor for downstream development. 


Stage 2: Converting Understanding into 
Customer Requirements 


In stage 2, the customer visit notes are analyzed to uncover the customer 
requirements (both explicit and latent) and the requirements are 
transformed from the language of the customer into the language of the 
company; from affective language into concrete statements. The vital 
few requirements are selected from the useful many and arranged in 
various combinations to create new insight. 


Stage 3: Operationally Defining Requirements 


In stage 3, characteristics of the vital few requirements are 
investigated with customers. Additionally, metrics are developed 
which will be used to measure quantitatively how well the 
requirements are met. Finally, all of the information and insight 
which has been developed is clearly and concisely displayed in one 
document. 


Stage 4: Generating Concepts 


In stage 4, the complex design problem is decomposed into smaller, 
independent subproblems. An exhaustive list of solutions (both feasible 
and infeasible) is created for each subproblem. Subproblem solutions 
are then combined to create solution concepts. 


Stage 5: Selecting Concepts 


In stage 5, the most promising solution concepts from Stage 4 are 
compared, in a structured process, against the customer requirements. 
The concepts which best fit the customer's requirements and the 
company's development capabilities are then selected for 
implementation. 
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stage I: 
Understanding 
the Customer's 
Environment 


The first of the five Concept Engineering Stages is Developing an 
Understanding of the Customer’s Environment. Critical to this stage is 
planning and scheduling of all downstream Concept Engineering 
activities. This stage consists of developing a plan for your team's 
exploration, doing the exploration, and using your team's interviews to 
develop a contextual anchor from the images of the customers’ 
environment. This contextual anchor is actually a KJ diagram of images 
of the customer's environment. The Image KJ is a link to the customer's 
real world and acts as a way to ground all future decisions in the 
customer's environment. 


As shown in the following figure, there are three specific steps included 
in Stage 1: Step 1, Plan for Exploration; Step 2, Collect the Voice of the 
Customer; and Step 3, Develop a Common Image of the Customer's 
Environment. 
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Concept Engineering Stages Stage 1 Steps 











Stage 1: Understanding the 


1: 


Step 
Customer's Environment Plan for Exploration 





Stage 2: Converting 
Understanding into Customer 
Requirements 






Step 2: Collect the 
Voice of the Customer 














Stage 3: Operationally 
Defining Requirements for 
Downstream Development 


Step 3: Develop 
Common Image of 
Customer Environment 





Stage 4: 
Generating Concepts 






Stage 5: 
Selecting Concepts 


The five Stages of Concept Engineering are shown above on the left and 
will be repeated throughout this document as a way to keep track of 
where each stage is in relation to the entire process. 
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Step 1: Plan for Exploration 


The purpose of Step 1 is for the team to discuss and understand the scope 
of the project and to develop a road map of future project activities. In 
this initial planning, your team agrees on the scope of the exploration 
theme, selects an exploration method, plans for and schedules the 
entire Concept Engineering process, and specifically plans for the data 
collection process. Planning for exploration will aid your team 
throughout Concept Engineering by supplying direction and purpose. 


Understanding the Scope of the Project 


Here the team must make a series of decisions defining the breadth of 
your exploration, starting with an exploration theme. 


The exploration theme is an important device for setting project scope. 
For example, the following potential exploration themes would result 
in projects of different levels of scope and complexity. 


A. “What are the most important customer requirements for 
flyfishing? 


Or: 


B. "What are the most important customer requirements for salt-water 
flyfishing?" 


Or: 


C. "What are the most important customer requirements for casting in 
a salt-water environment?" 


Or: 


D. “What are the most important customer requirements for a 
stripping basket?" 


Example A defines the scope as the entire sport of flyfishing. Example 
B narrows the scope to salt-water flyfishing. C narrows it even further 
to casting in a salt-water environment, and D is focused specifically on 
requirements for a stripping basket. 


In example B, note how the focus of the team must change if only a few 
words of the theme are changed. If the word "salt-water" is added, 
the focus changes significantly. 


The team should avoid limiting itself only to traditionally defined 
processes or services. Compare examples C and D above. Example D 
leads the team into the "solution space" of a stripping basket as an 





151 


answer to fishing line management problems. Example C, on the other 
hand, stays out of the "solution space" and remains in the "problem 
space" by directing the team to focus on the activity, not a solution to 
one of its problems. This subtle difference can significantly alter how 
the team develops its interview guidelines, thereby impacting what 
the team discovers. Whenever possible, do not limit your team's 
activity by adhering to conventional definitions of processes or 
practices. 


Select Exploration Method 


There are a number of methods for collecting data from your customers: 
in-person interviews, telephone interviews, laboratory observation, 
etc.. The figure below plots a number of different exploration methods 
on a two-dimensional map showing degree of intervention with the user 
vs. proximity to the user environment. 


Exploration Methods 


cus tomer ome 
t 
phonecaito Vis Ration with Sfeltainn with 
High ueer management 


Interven tion oe 
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with User 
unstruc tured intervis ws 
vieit to 


contextual hom emaker 


inquiry yw 


~~ 


procese participation 


gathering 
hu man antenna shop ev dence of 
perform ence iab a user behavior 
Low 


Intervention participant obeervation 


with User 
Sp A 
Far from User Close to User 
Environment Environment 


Unstructured interviews involve high intervention with the customer— 
an interviewer asking questions and follow-up questions based on the 
answers. These may occur either close to the customer's environment, as 
in a visit in the customer's office, or further from the customer's 
environment as in an interview conducted over the phone. 
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Process Participation involves some intervention and is by its nature 
closer to the customer's environment. Contextual Inquiry is a method 
used by Digital Equipment Corporation which involves sending 
engineers to the field to engage in a dialogue with the customers while 
they observe them using the product. 


Participant Observation involves low intervention and can range from 
close to far from the customer's environment. Observing customers from 
behind a one-way mirror is an example of this kind of research. 


Different methods are appropriate for different purposes. Keeping in 
mind that one purpose of Concept Engineering customer data collection 
is to extract images, that is, visualizations by customers using the 
product or service in their own environment, you should choose the 
method for exploration that best suits that need as well as other 
realistic constraints. In this document, we will focus mainly on in- 
person interviews at the customer's site, which include observations of 
their use environment. The major concepts and guidelines discussed, 
however, will be applicable to other methods as well. 


Planning for Concept Engineering 


It is important to have a well-designed plan of action for the entire 
Concept Engineering process. Concept Engineering consists of individual 
work and many team working sessions. Meeting coordination can be a 
powerful determinant of the team's momentum and eventual success. 
The team should agree upon a schedule and all members should plan for 
these meetings in advance. We recommend following Professor Shiba’s 
advice to schedule a project completion date first and then plan 
backwards, filling in the activities and dates. The schedule for 
Concept Engineering needs one or two weeks of dedicated time after 
data collection in order to process the data. During some steps, weekly 
meetings are needed. 


The Concept Engineering schedule will vary considerably depending 
upon the complexity of the team's project and the difficulty of reaching 
customers and scheduling interviews. However, all team-dependent 
activities should be scheduled into as short a time period as possible 
after the interviews are completed. The following guideline for task 
planning is a rough estimate of time required for each major task. The 
elapsed time is fifteen weeks. Some teams have completed Concept 
Engineering in fewer weeks; some have taken longer. 
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Planning for Data Collection 


Planning the specifics of data collection requires a thorough 
understanding of the types of customers in the markets that your team 
intends to explore, as well as an understanding of the exploration 
methods your team will use. Deciding the who, what, when, where, 
and how of data collection enables the team to consider how they will 
divide up market segments. 


How Many Interviews? 


Open-ended customer interviews are intended to explore the market 
and learn about customer needs. One of Professor Kawakita’s principles 
emphasizes the importance of using a small number of qualitatively 
rich cases. This principle is supported by the work of Professors 

Griffith and Hauser, who hypothesize that 10 to 20 interviews are 
sufficient if conducted with knowledgable customers. Twelve to fifteen 
interviews is a target that is realistic and not overly cumbersome. 
However, if you believe you have distinctly different market 
segments, you may need a minimum of 10 interviews per segment. 


Which Customers to Select? 
Diversity in selecting customers is important in order to explore the 


market broadly. A Customer Selection Matrix, shown below, is a 
helpful tool to search for diversity among the customers you visit. 


Customer Selection Matrix 


S a 
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Categories of Customers 


The categories down the left side of the matrix cover typical market 
segments. These may be geographically based, or based on any other 
category of market segmentation that is typical for your field of 
exploration. In the example above, fly-fishing markets can be 
segmented into: New England Salt-Water Flyfishing, Caribbean Flats 
Flyfishing, Salmon Flyfishing, Steelhead Flyfishing, Trout Flyfishing 
and Distributors. 


At the top of the matrix your team thinks about your customers in a 
slightly different way and lists categories of customers, for example, 
Lead-Users, Demanding Customers, Happy Customers, Unhappy 
Customers, Customers We've Had but Lost, and Customers We've Never 
Had. The categories across the top of the matrix will help ensure that 
you are not simply approaching those customers who are easiest to 
approach, but those who are, potentially, most useful to your Concept 
Engineering process. 


Lead Users, a concept developed by Professor Eric Von Hippel, have 
needs which typically foreshadow the general demand of the market. 
Additionally, they often have some prototype experience; that is, they 
have worked around existing product constraints to solve their problem. 
Therefore, Lead Users are particularly helpful in exploring the 
opportunities, as their prototype experience gives them a potentially 
greater ability to perceive and express needs. 


Using the Matrix 


Once the categories are listed, begin filling in customer names in each 
of the cells. Once all cells are filled in, select customers who are most 
appropriate for your project. 


It is not necessary that the team visit and interview a customer from 
each combination of categories; indeed, that would put the team well 
over the suggested 12 to 15 interviews. A general guideline from 
Professor Ed McQuarie, Santa Clara University, is to have three to four 
interviews in cells you consider most important and fewer, if any, 
interviews in the others. 


Who Conducts the Visits? 


Interviews should be conducted in pairs, preferably by two people from 
different functions in the organization, i.e., one from marketing and one 
from engineering. There are many reasons for using cross-functional 
interview teams. For example, the marketing person may be more 
likely to focus in on the customer's statements of needs; the engineer 
may focus on the customer's solutions to a difficult technical problem. 
Together they have a better chance of gathering a 360-degree view of 


156 


the customers world. Additionally, as engineers seldom spend time in 
the field, this is an excellent opportunity to put the engineer face to 
face with the customer. This opportunity to visit and explore the 
customer's environment can build the team’s empathy for the customer. 


Exploration Principles 


Market exploration through in-depth, open-ended customer interviews 
is a marketing research method which is different from methods 
intended to validate market hypotheses. Customer interviews can be 
used to develop hypotheses; traditional market research methods are 
still needed to test hypotheses, gather demographic data, etc. Asa 
different type of market research, customer interviews follow different 
guidelines than quantitative market research tools. Professor 
Kawakita has developed five guiding principles for collecting 
qualitative information: 


1. Take a 360-degree perspective. Walk around the situation from 
many different angles. Do not have a hypothesis about what you 
will hear; keep an open mind in order to explore broadly. 


2. Have a stepping-stone agenda. Do not schedule customer visits 
back to back. Allow for the possibility of an unexpected visit 
opportunity. Use an interview guide loosely; follow the path the 
customer creates. 


3. "By Chance". Utilize chances — but you can create chances if you 
are sensitive ; concentrate on the problem or opportunity in order to 
see chances to learn more. Louis Pasteur says, "chance favors only 
the prepared mind." 


4. Use your intuitive capability. Intuition is the tool of discovery, 
logic is the tool for proof. Human intuition has great capability to 
find somethind new. If your intuition is telling you to pursue a topic 
with a customer, follow your intuition. 


5. Qualitative vs. quantitative data. Numbers are not so important; 
cases and personal experiences are important. Diversity of insights 
is more important than the number of times you hear the same 
information. 


In addition to Kawakita’s principles, Peter Drucker’s work on sources of 
innovation is a helpful guide. Drucker states that innovation is most 
likely to be found in surprises, incongruities, and process holes. 
Customer interviews can be rich sources for discovering and exploring 
these innovation opportunities with a 360-degree perspective. 
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Developing the Interview Guide 


The Interview Guide is a small set of open-ended questions and sub- 
questions you can use as a guide during the interviews. It is only a guide, 
not a strict questionnaire that must be rigidly followed. It acts as a 
reminder to ensure that you cover all of the important topics. This is 
not to suggest that the guidelines restrict you from taking a different 
path during the interview. Indeed, it must be left to the interviewer to 
determine when there is an opportunity to veer off from the guidelines, 
either for broader exploration or to gather specific, detailed 
information. These opportunities usually occur in later interviews 
when the interviewers’ sensitivity and interview skill is increased. 


The questions which make up the Interview Guide can be generated ina 
number of ways. The Net-Touching process, outlined below, helps 
ensure that appropriate breadth and depth of coverage is developed. 


1. Brainstorm answers to the question, "What do we want to learn on 
our visits?” Write each thought on a 3"x3" label. (This could be 
done as individual work before the group meets). 


2. Prepare the meeting room as you would fora KJ. 


3. Have the team leader or facilitator randomly select a label, read 
it out loud, and post it to the board. They then ask other team 
members to provide labels of similar questions. 


4. When there are no more labels on the topic, write a title for the 
group, in the form of a question, which is one step higher on the 
ladder of abstraction. 


5. The team leader asks for a label on a new topic. 


6. Continue this process until all original labels are posted to the 
board, either in groups or as lone wolves. 


7. Continue grouping (by classification) the question-groups, writing 
titles in the form of questions until a small number (4 to 6) of groups 
are formed. 


Professor Shiba suggests ensuring that the questions cover the following 
categories. For any that are missing, add a new question. 


e Past problems or weaknesses of the product or service —these are 
useful in providing insight into the customer's perceptions. For 
example: What are the weaknesses or problems you've experienced 
with this (or this type of) product or service? 


e Current considerations associated with purchasing or using the 
product or service — these can be helpful in understanding the 
customer's expectations. For example: What do you think of when 
selecting this product or service? 


e Future enhancements of the product or service —this provides 
additional emphasis and detail on points previously discussed. For 
example: What new features might address your future needs? 
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e Scenes or images of the customer's use environment —these are 
essential for grounding all future development actions in the context 
of the customer's experience (Step 3). Some examples: What are 
the images that come to mind when you visualize the use of...? or, 
When you think about using this ... what are the scenes that come 
to your mind, or, If I had a video camera and recorded a scene of you 
using this product (or working in the lab, or struggling with a 
critical problem, etc.), what would | see? 


Compile the questions and sub-questions from the net-touching, adding 
any that come from reviewing the four categories of questions, into an 
interview guide. Think about a possible sequencing of the questions. 


interview Team 


Interviews should be done in pairs: that is, two people interviewing one 
customer. One of you is the questioner, the other takes verbatim notes of 
the customer's responses to questions. Notetaking does two things: it 
shows respect for the customer, and it captures the data. The 
questioner takes notes of observations and areas to explore. In general, 
the questioner stays engaged with the customer and does not take too 
many notes. The notetaker stays focused on writing the actual words of 
the customer (not a summary of the customer's thoughts) and should not 
ask too many questions. These roles of interviewer and notetaker can be 
reversed between interviews, but should not be changed during an 
interview. 


Practicing 


Practice interview sessions have proven to be powerful in developing 
interview skills. Role play an interview. Have one person as an 
interviewer, One as a notetaker, and one as a customer. Conduct the 
entire interview using the interview guide. Watch your body language 
to be sure you portray a message of interest throughout the interview. 
Debrief after the interview and discuss what worked well and didn't 
work well. Rotate roles so that each team member gains experience as 
a questioner and as a notetaker. 


Swim in Shaiiow Water 

Do a dry run of the interview process on familiar customers or internal 
people, and revise the guideline as necessary. This is a critical step; do 
not underestimate its importance. 


Doing Your Homework about Customers 


In order to show respect for the customer, before conducting visits , make 
sure that you are well briefed in the background of the customer, the 
products or services the customer purchases from your company, and 
additional information that may be meaningful to your visit. 
Understanding what your customer's business is and demonstrating that 
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understanding in the interview process will go a long way toward 
establishing a comfortable rapport for the visit. 


Tips 


Make sure to include the sales force as appropriate in planning and 
scheduling customer visits. 


Do as many practice interviews and shallow-water swims as time 
allows. The investment will pay off when you are doing customer 
interviews. The team will spend much time and effort on doing the 
interviews, and the success of the future product depends upon the 
data brought back from the interview. 


Completion Checklist 


At the end of Step 1 you should have: 


A schedule for the project 


A Customer Selection Matrix complete with the names of customers 
to be visited 


An interview guideline which has been tested 
Team members with training and practice in interviewing skills 


A list of who will visit whom 
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Step 2: Collect the Voice of the 
Customer 


The purpose of this step is to gather the customer data which will 
drive all subsequent Concept Engineering activities. 


A useful concept for this step is Professor Shiba's "Swimming in the 
fishbowl." It depicts the fishbowl as the user's (customer's) 
environment, with the swimmer (interviewer ) entering, Swimming 
around, exiting, and reflecting upon what was learned. In contrast, 
traditional market research looks at the market with developed 
hypotheses, essentially looking from the outside and measuring 
behavior in the fishbowl. Customer Visits and Contextual Inquiry are 
methods by which people who will make decisions about the product or 
service jump into the fishbowl, swim around and see what is going on, 
and then jump back out of the fishbowl to analyze what they saw. 


Swimming in the Fishbowl 






eat 
© 


So 
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Scheduling Visits 


The interviewing process requires the careful balancing of the 
interview schedule around customers’ availability and your needs. 
Generally the team members will have to adapt their schedule to the 
customers’. 


Prior communication with the customer is important to the success of the 
interview. A letter confirming the date, time, purpose, and the agenda 
should be sent to each customer. (It is also a nice touch to fax the 
interview guide to the customer a few days before the visit.) The degree 
to which you are open about your intention, and the degree to which you 
and your team honor the customer in a number of ways will determine 
the extent to which the customer will be open and honest in return. This 
effort at building rapport can be particularly valuable if you need to 
make additional contact. Honoring the customer at all times should be 
of highest priority. 


Schedule 60 to 90 minutes for each interview, but be flexible. If you 
schedule more than one interview at a site, allow for approximately 2 
hours between interviews. Even if you intend to conduct only one 
interview in a day, block off at least 90 minutes after the interview for 
debriefing with your interview partner. Preferably this will occur 
immediately after the interview concludes; conduct the debriefing in 
the parking lot if necessary. 


Conducting the Visits 


Visiting and interviewing customers is not a task to be taken lightly. 
Remember that every time you interface with a customer in the Concept 
Engineering process, it is for your education, not the education of your 
customer. Customer interviews are not opportunities for sales calls. It's 
your listening skills, not presenting skills, that are going to be tested on 
these visits. 


In fact, the listening required on a customer visit forces you to: 

e Be openand receptive to the customer's opinions and feelings. 

e Suspend all judgments. 

e Accept whatever the customer says 100%; never argue! 

One of your goals in the interview is to get beyond the first or obvious 
answers to the essential data. Often the customer's first response to a 
question is just the tip of the iceberg. Following up with probing 


questions is necessary to get to the bottom of the iceberg where the 
majority of the data is; explore the customer's thoughts in depth. 
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For example, the customer might give you a solution idea. Below the 
tip of the iceberg is the problem the customer is trying to solve. Explore 
this topic until you understand the nature of the problem in addition to 
the solution the customer has mentioned. The customer’s solution 
should be recorded because it may be useful in Generating Solution 
Concepts, but in order to learn about the customer's need, you must 
understand the problem hidden below the solution. 


Finally, remember to thank the customer for the opportunity they 
have given you to learn. 


Debriefing 


As was mentioned earlier, you should schedule 60 to 90 minutes after 
each interview to debrief. You may end up using the time to conduct a 
spontaneous interview. If this happens, you should still debrief as soon 
after the interview as possible. 

Follow these steps immediately after the interview: 

1. Make a copy of the notes. 


2. Discuss general observations for a few minutes. 


3. Read notes carefully, filling in gaps with the customer's actual 
words if you can recall them. 


4. Discuss the interviewer’s questions and follow-up questions. Note 
what worked well and didn't work well. 


5. Discuss and note insights about your customers, their environment, 
their needs, etc. that you gained from this interview. 


6. Think about improvements to the interview guide and note these. 
When you get back to the office: 
7. Share observations with other team members. 


8. Type up the verbatim customer notes, creating a transcript of each 
interview as soon as possible. 


Image Collection 


As discussed earlier, images are the scenes or descriptions of product use 
or the emotion associated with the product's use that are generated in 
the interview process. They may also include your observations. 


Read through each customer interview transcript and pull out the 
images of the customer's environment by looking at each customer 
statement and thinking about whether it is a past weakness, present 
consideration, future enhancement, or an image of the customer's 
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environment. At this point you are looking for images of the customer 
environment, e.g., images of the customer using the product or doing a 
task. Later you'll come back to the interviews and pick up the other 
customer voices which will be used to generate customer requirements. 


Write each image on a 3" x 3" Post-It Note with a black pen. Use the 
customer's actual words or your observations. Don't worry about 
including some statements that may also fit into another category (e.g., 
a past weakness). If in doubt, write it down. It is not unusual after all 
of the interviews are completed to have 100 or more images. 


Examples of images from the Stripping Basket Case Study are below. 






HAVING TO CAST 
QUICKLY. 


CASTING INTO A 
CHANNEL WHER E 
THE CURRENT IS 
HEAVY. 






















Rob UN DER YOUR 
ARM, STRIPPIN G 

BOTH HANDS INTO 
THE BASKET. 


Tips 


e Debrief as soon as possible after each interview and use this time to 
improve your interview skills and intuition about the customer's 
needs. 


e Remember to leave flexibility in your schedule to take advantages 
of the offers for plant tours, interviews which go longer than 
expected, etc. 


e Schedule time for the team to get together during the interview 
weeks to share observations and insights. 


Completion Checklist 


At the completion of Step 2, you should have: 
e Transcripts from each customer visit 


e Image statements written on 3x3 "Post-It" labels 
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Step 3: Develop a Common 
Image of the Customer 
Environment 


The purpose of this step is for the team to create a common mental map 
of the customer's environment. This map is the primary device for 
keeping the team grounded in the customer's use environment. The 
Multi-stage Picking-out Method was developed by Professor Kawakita 
as part of the KJ process. It is used to select the best images collected in 
step 2 for subsequent use in creating the Image KJ. The image KJ follows 
from the work of Professor Ohfuji and colleagues, and is a critical 
component of the requirement statement development process described 
in Step 4. 


Gather Image Statement Labels 


During Step 2, Collecting the Voice of the Customer, image statements 
identified from each interview were transcribed onto 3"x3" Post-It 
labels. These labels should be collected and posted on a large table or 
to a wall. Use the Multi-stage Picking-out Method (MPM), COM 
Document 5P, to reduce the number of images to between 24 and 30. This 
number is a manageable size for the Image KJ. When the number of 
images increases toward 30, the time required for the KJ increases 
dramatically. 


MPM Selection Criteria 


The following criteria for selecting images for the Image KJ should be 
displayed prominently next to the MPM work area and repeated before 
the start of each round. 


1. Images should reflect the personal experience of a user. They are 
often written in first person. For example: "I come home from 
fishing and throw my basket by the porch stairs and there it 
Stays.” 


2. Images should be capable of standing by themselves without 
amplification or explanation from the author. For example: 
"Fishing from the platform on the bow of the boat." 


3. Diversity of images is essential. It may be better to select an image 
label of slightly inferior quality according to Criteria 1 and 2 in 
order to obtain coverage of an area not yet covered. 
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select Most Significant Images 


MPM is a tool which involves the team in choosing the vital few from 
the useful many. The focus is on picking up strengths, not eliminating 
weaknesses. Select a target number of Image labels usually around 24, 
and a cutoff number (1/3 more than the target). The theme is usually: 
"What are the scenes or images of the customer's environment?" The 
theme and selection criteria should be posted next to the labels. 


We assume familiarity with the use of MPM. See CQM Manual 5P for 
detailed instructions. 


In the initial ("multi-dot") rounds, every participant can select any and 
all the labels they want to keep by marking a small dot on the lower 
right corner of the label with a red pen. There is no time limit to any 
round or limit to the number of labels you can mark. However, it is 
important to recognize that the number must be reduced and you need to 
be successively more selective. The initial rounds are completed when 
the number of labels remaining is approximately equal to the cutoff 
number (around 32). 


In the final ("limited-dot") rounds, you can select only a predetermined 
number of labels. This limit is usually calculated by dividing the 
target number by the number of participants. In these rounds, you select 
in order, each label being read out loud before the next in line selects. 
When everyone has selected their allotted number of labels the target 
number should have been reached and the selection process is complete. 


Create Image KJ 


The Image KJ is slightly more difficult than a traditional KJ. The KJ 
process, as outlined in the CQM KJ manual (Document 2P), provides the 
basic KJ steps. Several additional guidelines are provided for Image 
KJs: 


e The theme for the image KJ might be: "What are the scenes or 
images of the customer's environment." 


e The scrubbing step should not change the words on the label. If a 
label requires clarification the author should provide additional 
context by referring to the appropriate interview transcript. 


e The titles to the groups should continue to reference the customer 
and their environment. Titles which contain the product as the 
subject tend to be requirement or solution oriented rather than 
descriptions of the environment. 


e There is no need to vote on the most important groupings. 
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Reflection 


Upon completion of the Image KJ, it is time for your team to pause and 
reflect upon what you have learned. It may be that you have learned 
you still need information from other customers who were excluded, or 
that you learned something using this process that may not have been 
uncovered otherwise. You should also reflect on the process and how it 
might be improved the next time. 


Tips 


¢ Avoid selecting Images which are highly abstract and don't give 
you a good picture of the customer's use environment. 


e Images which contain or evoke emotion are preferable to those 
which are purely descriptive. 


¢ Watch for a tendency to migrate from customer to product 
Orientation during title development. (Titles will start looking like 
customer requirements instead of images of the customer's 
environment). 

e Be careful of clarification discussions which are not anchored in the 
interview transcripts; they can lead to misinterpretations. 


Completion Checklist 


At the end of this step, the Image KJ should be complete. 
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stage 2: Converting 
Understanding 

Info Customer 

Requirements 






The second of the five stages in Concept Engineering is Converting 
Understanding into Customer Requirements. This stage distills what 
was learned from the customer exploration into a small set of well 
understood key customer requirements. The Image KJ] developed in Step 
3 will be used as a contextual anchor to ensure that the requirement 
statements developed are consistent with the customers’ environment. 


There are three steps in Stage 2. In Step 4, the transformation method 
converts the customer's language, often laden with emotion, into a 
requirement statement better suited for use in downstream development 
activities. Step 5 uses the Multi-stage Picking-out Method to identify 
the vital few requirements from the useful many. Step 6 employs the KJ 
method to develop new insight and team consensus regarding the 
relationships among requirements. 
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Concept Engineering Stages 







Stage 1: Understanding the 
Customer's Environment 










Stage 2: Converting 
Understanding into Customer 
Requiremen 









tage 3: Operationally 
Defining Requirements for 
Downstream Development 






Stage 4: 
Generating Concepts 


Stage 5: 
Selecting Concepts 
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Stage 2 Steps 


Step 4: Transform Voices 
into Customer Requirements 


Step 5: Select Most 
Significant Requirements 


Step 6: Develop Insight into 
Customer Requirements 


Step 4: Transform Voices into 
Customer Requirements 


The purpose of this step is to develop customer requirements which are 
unrestricted and unambiguous. Requirement statements should reflect 
the customer's need, not potential solutions to the need which 
inadvertently restrict the requirement. Requirement statements should 
minimize ambiguity. When requirement statements are expressed in 
ambiguous terms that allow for diverse interpretations, subsequent 
development activities can be both inefficient and ineffective. The 
methodology links each selected voice with an appropriate image or 
images from the Image KJ developed in Step 3. Key items (key 
thoughts or essential ideas) from the image-voice association are 
extracted and converted into requirement statements through an 
iterative refinement process. Translation guidelines are used to 
develop requirement statements which are unrestrictive and 
unambiguous. 


Voice 1 See gems: Key ltem—> Requirement 






; er et et we Associated oo 08 0% oe 








Key Item—_> Requirement 
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The Customer Requirement Worksheet 


To assist you in recording Customer Voices and Images, extracting Key 
Items, and developing Customer Requirements, you use a simple 
worksheet. A small-scale example can be seen below, and a full-page 
version is included in Appendix F. Using this worksheet not only helps 
you generate requirements but provides you with an audit trail that can 
be useful for clarifying discussions about requirements. 


The Worksheet 


| ## |customerVoice| image | Keytem | Cus. Rea! 


The first column is simply for the purpose of numbering the voices. The 
next column comes directly from the interview notes. A "Voice" is 
defined as an individual thought, idea, or statement that is to be 
considered and can be understood on its own merits. Not every word 

that is captured in the interview notes is transcribed here. Only the 
direct language that is meaningful as a unique thought from that 
interviewee is defined as a voice. There may be 50 or more voices from a 
single interview. 





These guidelines will help you make good use of the requirements 
worksheet: 


e Begin to use the worksheets early in the process. 


e Leave blank rows between voices on the worksheet so you can 
expand the number of Key Items and Customer Requirements per 
voice. 
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Selecting Voices for Transformation 


Each interview can produce dozens of customer statements (voices) 
appropriate for transformation into customer requirement statements. 
Ideally, each voice will be transformed in order to maximize the return 
from the exploration investment. However, on occasion this will not be 
feasible. In these instances there are several alternatives for selecting 
a subset of the total voices for review. 


The preferred alternative is for every voice to be read, linked to 
appropriate images, and have key items determined. If the key item 
has been previously identified and transformed into a duplicate 
customer requirement statement, no further action on this voice is 
required. 


Another alternative is for all the voices to be reviewed and categorized 
according to common criteria, e.g., using the Net-Touching method. The 
voices which are the best exemplars of each category are then selected 
for transformation. 


Finally, another alternative is for the team to identify a list of 
screening factors for voice selection. For example, statements regarding 
regulatory requirements might be excluded if it was already known 
that all regulatory requirements would be met. Only the voices which 
passed the screening criteria would be transformed. In determining the 
screening factor list, it is important that each person selecting the 
subset of voices clearly understands the screening criteria. 


Identifying the Contextual Anchor 


Customer requirements must be related to the customer's actual 
environment. To ensure that customer requirement statements reflect 

the customer's experience, each selected voice is associated with an 
image from the Image KJ developed in step 3. The voice can be linked 
at any level of abstraction, i.e. first-, second-, or third-level titles, or to 
an actual data label; most often the linkage is made at the first-level 
title. It is often the case that a voice can be linked to more than one 
image. Creating these additional linkages is encouraged, as the new 
patterns of association can be a source of discovery. 


Extracting the Key Item 


Often it is not clear what the voice, in the context of the image, really 
means. Determining the key items from the marriage of the voice and 
image serves as a bridge to assist the development of the customer 
requirement statements. One approach to determining the key items is 
to start by systematically identifying significant words in the voice 
and making a corresponding key item. Another method is a free- 
association approach in which the first things that come to mind are 
used as the starting point. Key items are subjected to rapid iterative 
improvement in the requirement statement development process. 
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Creating the Requirement Statement 


The customer requirement statement must be stated in the language of 
reports without being unnecessarily restrictive or ambiguous. If the 
requirement is clear and concise it will allow later development 
activities maximum design flexibility while minimizing the risk of 
misdirected effort. 


Based on our experiences, we developed three Translation Guidelines 
and a Transformation Worksheet, Appendix (F). The guidelines are 
presented in precedence order, i.e. guideline number 1 is more important 
than guideline 2, and so on. 


Translation Guideline 1: Avoid Statements of Means 


Requirement statements that contain solutions severely restrict the 
freedom of designers to consider alternatives. The team should work 
hard, therefore, to avoid statements of means — "how to" language — 
when writing requirements statements. 


Customer's Voice Image Key Item Re 
"Quick release basket so it|"Fishing from the |1. The basket is 
doesn't get in the way platform onthe [released easily. 
when moving around the {bow of a boat.” 
boat after a fish." 


Guirement Statement 
(-) The basket has velcro 
fasteners. 

























(+) The basket is 
releasable with one hand. 


The better (+) requirement statement clearly states the requirement 
without restricting potential solution alternatives. The weak (-) 
statement restricts the solution alternatives to velcro. 


Translation Guideline 2: Avold Abstract Terms 


Requirement statements which contain abstract or vague terms allow 
for multiple interpretations of the customer's requirement. This can 
result in development activities which are inconsistent with the 
customer's actual requirement or at cross-purposes with each other. 


Customer's Voice Image Key Item Requirement Statement 
"Durable, material made |{"I come home from |1. The basket must](-) The basket is durable. 
out of cane won't last; fishing and throw |withstand the 
plastic will last longer my basket by the |environment. (+) The basket is 


than me." porch stairs and saltwater resistant. 
there it stays" 2. The basket must}(+) The basket with- 
last several stands exposure to 
seasons. the sun. 





The better (+) requirement statements clearly state the requirements for 
environmental factors. In the weak (-) requirement statement, the term 
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"durable", permits many possible interpretations. In this project, 
additional requirement statements were also developed for durability 
as it relates to rough physical handling. 


Translation Guldeline 3: Use muitil-valued thinking 


Design constitutes a continual series of tradeoffs. Requirement 
statements which are multi-valued imply a range of performance and 
allow the developer flexibility in design. Requirement statements 
which are not multi-value oriented are "0/1," or "here/not-here" in 
orientation. Requirement statements which are written in a fashion 
which implies the requirement is either totally included or excluded 
unnecessarily restrict design freedom. 


Customer's Voice Image Key Item Requirement Statement 
"How the water spills out |"Tides, waves, 1. drainage (-) Water does not 
of it; drainage." seaweed" accumulate in the basket. 





(+) Water drains freely 
from the basket. 


The better (+) requirement statement implies a performance measure 
(time) and range of performance (quick to slow). The weak (-) 
requirement statement addresses whether water does or does not 
accumulate in the basket. 


Grammar Guidelines for Writing Requirements 
Effectively 


Professor Shiba has taught that the subject of the requirement 
statement must relate to the product or its attributes. Additionally, 
Strunk and White’s classic The Elements of Style identifies some 
principles of composition that are essential to keep in mind during 
requirement statement development. 


"Be clear. When you say something, make sure you have sald it.” 
Using a simple sentence structure, subject-verb-modifier helps you to be 
clear. For example: 

"The basket adjusts to placement on body." 
Instead of: 

"The basket position can be adjusted to accommodate various 


casting styles. " 


“Place emphatic words of a sentence at the end. The proper 
place in the sentence for the word or group of words that the writer 
desires to make most prominent Is usually the end.” 


For example: 
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“Line is tangle-free." 
Instead of: 
"Tangle-free line is essential" 
“Use the Active Voice. The active voice is more direct and 
vigorous than the passive.” 
For example: 
"You can release the basket with one hand." 
Instead of: 


"The basket can be released with one hand." 


“Put statements in positive form. Make definite assertions.” 


For example: 


"Water drains freely,” 


Instead of: 


" 


" Water does not accumulate in the basket. 


Transformation Tips 


e Develop the requirement statement by rapidly making successive 
improvement of the key items and requirement statements rather 
than attempting to write a statement which addresses all of the 
thoughts on the first try. 


lst Customer 
Y SEBO acl eee Renee Attempt 
2nd Key Item attempt | —t» ond Customer 
Requirement Attempt 





3rd Customer 
Requirement Attempt 


e Use of the word "not" is an indicator of a requirement statement 
which is weakness-oriented not strength-oriented. Positive- 
orientation is preferred as the absence of weaknesses is not strength. 


e Use of the words "must" and "should" usually indicate a lack of 
multi-valued orientation. 
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e Use of the word "and" usually indicates that more than one 
requirement is contained in a single statement. 


¢ Itis not unusual for a single key item to produce more than one 
requirement statement or for multiple key items to generate only one 
requirement statement. 


¢ Use of the worksheet will create a permanent audit trail that has 
proven very valuable in subsequent clarification discussions. 


¢ You should block out a couple of dedicated hours for transformation 
work, and then leave it; come back to it at another time witha 
fresh perspective. Don't force it. 


¢ Requirement statements can be developed by individuals or in small 
groups. We recommend that teams work in small groups (2 or 3 
people) initially. 


e When a group works on developing customer requirement statements 
together, members may find that there are different 
interpretations of the key items in a voice, or different 
interpretations of the image to which the voice relates. This may 
be a result of different members having heard different comments 
from their customer interviews or simply based on their functional 
backgrounds, e.g., marketing vs. engineering. Different 
interpretations are opportunities for creativity — for finding some 
hidden requirements. Pursue all of these opportunities for 
requirement development. You will have a chance later in the 
process to validate the requirement statements with customers. 


e Occasionally an appropriate image cannot be located on the Image 
KJ. It is acceptable to use an image which did not make the final 
round of MPM , and is not on the Image KJ as the anchor. 


e¢ Any solutions discovered in reviewing visitation transcripts should 
be identified and segregated for use in Stage 4: Concept Generation. 


Completion Checklist 


At the completion of this step all selected voices should be transformed 
into Customer Requirement statements using the Translation 
Worksheets. Well-written requirement statements should: 


e Have simple, subject-verb-modifier, sentence structure. 
e Be free of specific solutions. 


e Be ata concrete level of abstraction through the use of definite, 
specific language. 
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Step 5: Select Most Significant 
Requirements 


The purpose of this step is to determine a manageable set of key 
customer requirements to focus on. The empathy for the customer which 
has been developed during prior steps is brought to bear on the selection 
of the most significant Customer Requirements. The Multi-stage 
Picking-out Method (MPM )will be used to select the most significant 
requirements. The requirements selected in this step will drive all 
further effort in Concept Engineering. 


Gather Customer Requirement Statements 


The final customer requirement statements which were developed in 
step 4 (last column of the worksheet) should be transferred onto 3"x3" 
"Post-It" labels and placed on a large table or wall. Due to the large 
number of requirement statements (300 or more is not unusual) it is useful 
to collocate similar requirement statements. 


To initiate the collection process, the facilitator randomly picks one 
customer requirement statement label. It is read out loud and placed on 
the wall/table. Without discussion, anyone else who feels he or she 
has a label which is similar hands it to the facilitator, who places 
these labels, without reading or comment, in the vicinity of the first 
label. When all offered labels are posted, the facilitator randomly 
selects another label reads it out loud and places it on the wall or table. 
Again, anyone who feels he or she has a similar label will have these 
posted by the facilitator without comment. This continues until all 
labels are posted to the board. No attempt is made to formally 
categorize the groupings. 


Define the Selection Criteria 


Discuss selection criteria before conducting the MPM. It is very 
important that the selection criteria be focused on the customer's 
requirements, not on solutions or features which are personally 
attractive to members of the group. Additionally, because only the 
twenty to thirty most significant requirements are ultimately selected, 
an emphasis on diversity is important. Finally, as all labels will be 
reviewed and refined in Step 6, focus on the underlying thought, not 
necessarily the words of the requirement statement. 
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select Most Significant Customer Requirement 
Statements 


The selection process should be effective and efficient. MPM is a tool 
which involves everyone equally in systematically choosing a small 
number of items from a large number. The goal is to pick up strengths not 
to eliminate weaknesses. A target number of requirement statements is 
selected (usually around twenty-four) and a cutoff number (1/3 more 
than the target number) is also determined. The theme is written and 
displayed prominently next to the labels. The theme is usually: 
"What are the customer's most significant requirements for 

?" The selection criteria should also be posted next to 
the labels. 


In the initial (“multi-dot") rounds, you can select any and all the labels 
you feel are significant by marking the lower right corner of the label 
with a marker. There is no time limit to any round or to the number of 
labels you can mark. However, it is important to recognize that the 
number must be reduced and that you have to be successively more 
selective. The initial rounds are completed when the number of labels 
remaining is approximately equal to the cutoff number. 


In the final ("limited-dot") rounds, you can select only a predetermined 
number of labels. This limit is usually calculated by dividing the 
target number by the number of participants. In these rounds, you select 
in order. After a label is selected, it is read to the group before the next 
person in line selects. When everyone has selected their allotted 
number of labels the target number should have been reached, and the 
selection process is completed. 


Selection Tips 


e When organizing the requirement statements at the beginning of 
the MPM, do not title the groups; omitting titles will prevent 
premature classification of requirement categories. 


e Emphasize that the MPM rejects are not thrown away and can be 
used in later development activities. This selection process is 
designed to select those key requirements which will define the 
product in the marketplace. 


e Discourage discussion during the early rounds; this prevents 
valuable energy and time from being spent on requirement labels 
that will not survive the pick-up process. 


e When discussion regarding the meaning of a label does occur, go 
back to the worksheets to ensure that the requirement statement is 
placed in the proper context of customer voice and image. 


e During the MPM it can be useful to have someone, who is not 
involved in the selection process (perhaps a facilitator) separate 
the rejected requirement labels into groups with a common theme. 
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e During the final rounds of the MPM it may prove useful if the final 
rejects are kept handy for review during the "check for omission" 
stage of the Requirement KJ in Step 6. 

Completion Checklist 


Upon completion of this step twenty to thirty key requirements should 
have been selected for subsequent investigation. 
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Step 6: Develop Insight into 
Customer Requirements 


This step forms the foundation for creative concept generation. Using 
the requirement statements selected in step 5, the essence and structure 
of the Customer Requirements are abstracted and examined using the KJ 
method. The KJ encourages the creation and examination of a myriad of 
requirement relationships, which facilitates the opportunity for 
creativity and insight. 


Review Customer Requirement Statements for 
Clarity 


This is the first point at which each requirement statement, one at a 
time, will be systematically reviewed by everyone. If diverse 
interpretations exist, the Transformation Worksheet should be 
reviewed to ensure consistency with the original customer's voice and 
image. The requirement statement should be rewritten for clarity if 
required. If a everyone agrees on the interpretation of the requirement 
statement, discussion is not necessary. 


Create Relationships 


Follow the KJ process as outlined in the CQM KJ manual. The theme for 
the KJ might be: "What are the most significant customer requirements 
for 2 


Reflect 


Before progressing to the next stage of Concept Engineering, reflect on 
what has been learned to this point. Are there any surprises or 
inconsistencies which might constitute opportunities for innovation? 
What additional information would be useful to have? What would be 
done differently if it could be done over again? Is there a need to go 
back for additional exploration before moving forward? 


Tips for Requirements KJ 


e Inthe check for omission step, after the first round of grouping, refer 
to the Image KJ and late-round rejects from the requirement MPM to 
help identify possible key omissions. 


e The first round of grouping should take an hour or so if all of the 
plausible relationships are to be considered. Don't rush the 
grouping. 

e The concept of taking only one step at at time up the ladder of 
abstraction must be enforced in the title making process. 
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e Titles should only address the intersection, not the union of the 
labels in the group. 


Completion Checklist 


At the completion of this step the Customer Requirements selected in 
Step 5 will be structured in a Customer Requirements KJ. 
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Stage 3: 
Operationalizing 
What You Have 

Learned 





We have heard repeated complaints that development concepts 
changed over time, even after managers had "signed off" in various 
review processes. Clearly, the product development process is 
inefficient if people do not agree on what is important or expected. 
Stage 3, Operationalizing What You Have Learned, should ensure that 
the key Customer Requirements are clearly, concisely, and 
unambiguously stated in measurable terms. 


At the completion of this stage, the team will have developed a 
Quality Chart and Operational Definitions. By reviewing these 
documents, anyone associated with the development effort can easily 
see the relationships between Customer Requirements and can 
determine their relative priority. They can also find specific 
measurement techniques for assessing conformance to the requirements. 
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Concept Engineering Stages 








Stage 1: Understanding the 
Customer's Environment 











Stage 2: Converting 
Understanding into Customer 
Requirements 








tage 3: Operationally 
Defining Requirements for 
Downstream Development 





Stage 4: 
Generating Concepts 


Stage 5: 
Selecting Concepts 





Stage 3 Steps 


Step 7: Develop and 
Administer Questionnaires 


Step 8: Generate Metrics for 
Customer Requirements 


Step 9: Integrate 
Understanding 


Step 7: Develop and Administer 
Questionnaires 


The product development team will enter Step 7 with a well-focused 
list of key requirements. The main objective of Step 7 is to help the 
design team develop a feeling for which requirements should receive 
the greatest attention from the team in later design efforts. 

We will use three procedures to achieve this goal: 


e The Kano Questionnaire, which characterizes the nature of the 
requirements 


e The Self-stated Importance Questionnaire, which measures the 
importance of the requirements 


e The Critical Requirement Questionnaire, which selects the critical 
groups of requirements. 


In general, the steps to follow for all questionnaires are as follows: 
1. Develop the questionnaires 

Test the questionnaires and revise if necessary 

Administer the questionnaires to customers 


Process the results 


na —- W NN 


Analyze the results 


This section emphasizes the Kano Questionnaire, a tool we are just 
learning to use. We briefly address the other two questionnaires first. 


Self-Stated Importance Questionnaire 


According to research by Professor Hauser at MIT, The Self-Stated 
Importance Questionnaire can be helpful in understanding the relative 
importance of each requirement for the customers. 


Method 
Constructing the Self-Stated Importance Questionnaire is straight- 
forward: 


1. For each of the requirements (the black-level labels) on the 
Requirement KJ, construct a question in the following general 
format: "How important is it or would it be if: [requirement x]}?" 
For example: "How important is it or would it be if the line is cast 
without drag?" 


2. Provide a scale on which customers can mark their responses, from 
“Not at All Important", to "Extremely Important." 
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Stripping Basket Self-Stated Importance Questionnaire 


Not at All Somewhat Very Extremel 
Important Important Important Important Importan 


How important is it or would it be if: 
the line is cast without tangles? 1 2 S “ 


How important is it or would it be if: 
the water drains from the basket freely? 


How important is it or would it be if: 
the basket color does not fade over time? 


How important is it or would it be if: 
the basket can be attached at different 
body positions? 





Critical Requirements Questionnaire 


As we indicated in the discussion of the Requirements KJ, we recommend 
having customers perform the voting step. If you wish to have 
customers vote on your Requirements KJ, it is convenient to gather their 
input at the same time you distribute the other questionnaires. 
Therefore, you may also want to construct a Critical Requirements 
Questionnaire. The only drawback is potentially overloading your 
customers with materials. You must make a judgement call about how 
much material they can handle. 


Method 


1. Make a list of the red-level labels (black labels that are lone 
wolves at the red-level belong in the list too). If you wish, you can 
provide additional detail by indenting the text from the black- 
level groups under the appropriate titles. 


2. Instruct the customer to pick the 3 most important items from the 
red-level list and rank them in priority. 


Kano Questionnaire 


Professor Noriaki Kano has developed a tool which helps us 
understand the relationship between the fulfillment (or non- 
fulfillment) of a requirement and the satisfaction or dissatisfaction 
experienced by the customer. Kano achieves this by classifying each 
customer requirement into one of 5 categories: must-be, attractive, one- 
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dimensional, indifferent, and reverse. The definitions for these terms 
and a synopsis of the underlying theory are presented in Appendix D, 
“Understanding Kano Theory.” We suggest that first-time readers 
review this appendix before proceeding. 


The Kano Questionnaire is only one tool for setting development 
priorities. While it provides a unique perspective on customer 
requirements, it is probably most effective when interpreted as a guide 
to be combined with other requirement assessment methods. 


Method 


In constructing the questionnaire you will form two questions for each of 
the requirements appearing in your Requirements KJ. The first question 
will always refer to a situation in which the requirement is met, and 
will be worded in a format similar to the following: "If the [product] 
satisfied [requirement x], how would you feel?" This is called a 
functioning question. The second question will always refer to the case 
where the requirement is not met. This is called the dysfunctioning 
question, and is worded in a format similar to the following: "If the 
[product] did not satisfy [requirement x], how would you feel?" 


Developing the Kano Questionnaire 


Divide the requirements from your KJ among team members.Write a 
functioning and dysfunctioning question for each requirement using the 
guidelines below: 


e You may have to step down the ladder of abstraction to construct a 
clear question. Avoid straying from the original intent of the 
customer requirement statement. Refer to the Requirement KJ and 
Translation Worksheets if necessary. 


e Beware of polar wording in the question pairs; multi-valued 
orientation is preferred. Consider this functional question: "If line 
placed in the basket stays in it until cast, how would you feel?" 
instead of wording the dysfunctioning question : "If line placed in 
the basket falls out before casting, how would you feel?"; It would 
be preferable to ask, "If some line placed in the basket falls out 
before casting, how would you feel?" 


e. Don't try to cram several thoughts into one question. You want to 
know which question the respondent is answering. If a particular 
requirement contains more than one thought, use more than one 
Kano question. If you opt to generate more than one question for 
some requirements, you should keep in mind the need to keep the 
survey as short as possible. 


e¢ When you have functional and dysfunctional wordings for each 


question, put the five standard answers beside each question as 
follows. 
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Stripping Basket Kano Questionnaire 


8a.If the line does not move . I like it that way. 
around in the basket, how do | 2- It must be that way. 
. I am neutral. 
. I can live with it that way. 
. I dislike it that way. 


you feel? 


8b.If the line moves around in the | 1. I like it that way. 

basket, how do you feel? . It must be that way. 
. | am neutral. 
. I can live with it that way. 
. I dislike it that way. 


9a.If line placed in the basket . I like it that way. 

stays there until casting, how do_ | 2. It must be that way. 

you feel? . am neutral. 
. I can live with it that way. 
. I dislike it that way. 


9b.If some line placed in the . I like it that way. 
basket comes out before casting, | 2- It must be that way. 


how do you feel? -Tam neutral. — 
. I can live with it that way. 


. I dislike it that way. 


10a.If line in the basket is tangle | 1. I like it that way. 

free, how do you feel? . It must be that way. 
. I am neutral. 
. I can live with it that way. 
. I dislike it that way. 


10b.If line in the basket . I like it that way. 
occasionally tangles, how do you | 2- It must be that way. 
feel? . I am neutral. 
. I can live with it that way. 
. I dislike it that way. 


lla.If line gathers naturally in the | 1. I like it that way. 

bottom of the basket, how do you | 2. It must be that way. 

feel? . I am neutral. 
. I can live with it that way. 
. I dislike it that way. 


11b.If line does not gather . I like it that way. 
naturally in the bottom of the . It must be that way. 


basket, how do you feel? -Tamneutral. — 
. Ican live with it that way. 


. I dislike it that way. 





Testing the Questionnaires 


Since the questionnaires will be sent to many customers, it’s important 
that they be understandable. We recommend testing all of the 
questionnaires internally before distributing them to customers. A test 
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run will help identify confusing wording, typographical errors, or 
unclear instructions. 


Refining the questionnaires may require iteration. These guidelines can 
help you test your questionnaires effectively: 


I 


Have the team answer the questionnaire first. Think of a customer 
and try to predict their responses. Note which questions you think 
your customer may not understand. 


Select people inside your company to answer the questionnaire, and 
administer it to them. Select from a variety of backgrounds, e.g. 
senior managers, development engineers, marketing personnel, etc. 


Revise the questions and retest. 


Administering the Questionnaires 


There are many details to consider in administering the questionnaires. 


Select the customers you would like to survey. We suggest returning 
to the customer selection matrix to develop a target list, applying 
the same criteria discussed in that step. In order to assure a 
statistically significant sample, most teams poll considerably more 
customers than were interviewed. Remember that not all customers 
will respond (for more information, see Appendix C, "Additional 
Hints on Administering Surveys"). 


Decide on the medium to be used. You might use telephone (voice or 
fax), face-to-face presentation, mail, or other means. The most 
common method is through the mail. If you plan to use the mail, 
write a letter of introduction which explains the purpose of the 
survey and includes directions for the customer. See Appendix C for 
an example. 


Collect demographic data which will enable you to distinguish 
potential market segments if they exist. Consider collecting 
information on company and personal characteristics, familiarity 
or experience with product, use of competitors products, etc. 


Include instructions for filling out the questionnaires. This is 
particularly important for the Kano questionnaire since it is likely 
to be new to customers. 


If you are using the Self-Stated Questionnaire in addition to the 
Kano Questionnaire, use the same sequencing of questions to make 
comparing the results of the two questionnaires easier. 


Send out the surveys. Keep a log of customers to whom the surveys 
were sent, along with the date. This will help to avoid 
duplication if you choose to expand your sample later, and to follow 
up if necessary. 


Record responses as they arrive. 
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Processing the Kano Results 


1. Translate each response to the question pair into a quality element 
code. To do so, look at each pair of questions on the Kano survey, 
and note the Functioning and Dysfunctioning answers for each 
question. Refer to the Two-dimensional table of evaluation, shown 
below, and locate the cell at the intersection of the Functioning and 
Dysfunctioning responses. 


Two-dimensional table of Evaluation 
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Customer Requirement is: 





A: Attractive O: One-dimensional 
M: Must-be QO: Questionable result 
R: Reverse I: Indifferent 


To illustrate how to score the questionnaire, question 9 from the 
stripping basket questionnaire is shown below. 


9a. If line placed in the basket 1. I like it that way. 
stays there until casting, how do_ | 2. It must be that way. 
you feel? 3. I am neutral. 
4. I can live with it that way. 
5. I dislike it. 


9b.If some line placed in the 1. I like it that way. 
basket comes out before casting, | 2: It must be that way. 


how do you feel? 3.lam neutral. 
4. I can live with it that way. 


5. I dislike it. 





If the customer responded to part (a) by circling #1 “1 like it that 
way” and responded to part (b) by circling #5 “I dislike it’, then 
look at the intersection of the first row and the fifth column, you 
will find an "O," indicating that the customer views the amount of 
line which falls out of the basket as a One-dimensional element. 


2. Record the requirement codes (i.e., A, M, O, R, Q, or I) on the 
tabulation matrix. An example of the tabulation matrix appears 
below. 


2 


3. Repeat the above steps for all the questions on the survey for each 
customer who returned the survey. 

4. Inthe far right column of the tabulation matrix, assign a grade to 
each of the requirements. The grade should be whichever code 
appears most often in the responses for that requirement (i.e., it is 
the mode of the responses). See the next section and Appendix D for 
more sophisticated analysis of Kano results. 


Evaluations of C.R. for Stripping 
Basket Kano Questionnaire 
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Kano Response Analysis 


Several benefits are obtained from analyzing Kano data: 


e Gaining a better understanding of requirements 


193 


e Prioritizing requirements for development activities 


e Distinguishing market segment characteristics. 


In the discussion of the underlying theory, Appendix D, we addressed 
the qualitative distinctions between the five types of quality elements. 
Some screening rules were provided in the previous section. This section 
will focus on techniques we can use to interpret the data for prioritizing 
future development activities. Remember that the purpose of these 
questionnaires is to better understand the characteristics of the 
customers’ requirements. The responses to these questionnaires should 
be seen only as a guide —no exact answer is provided as to which 
features must be included in the product, or which requirements need not 
be fully satisfied. 


Alternative Analysis Approaches 


A simple way to rank the requirements is to score each according to the 
mode, the most frequently occurring dimension in each row of the 
tabulation matrix. Thus, a requirement voted Must-be by 43% of 
respondents and Attractive by 38% of respondents would be interpreted 
as a Must-be requirement. 


You may want to investigate the response distribution beyond the 
simple mode, i.e., looking at the second most frequent dimension for 
each requirement. For example, take a case where there are two 
questions and fifty responses to each. Thirty customers rate the first 
requirement Attractive and twenty rate it Indifferent. On the second 
requirement, thirty customers again rate it Attractive, but the 
remaining twenty rate it Must-Be. In this case, it is likely that the two 
requirements should be treated differently by the team. The second 
requirement should receive higher priority from the team. 


We suggest constructing a spreadsheet with columns for the first, 
second, and third most frequent responses. The next step is to rearrange 
the rows into groups according to the following order: Must-Be’s first, 
One-Dimensionals second, followed by Attractives, Indifferents, and 
Reverse requirements. 


The Self-Stated Questionnaire responses can be helpful in further 
sorting the Kano responses. One way to order the requirements within 
each group is by importance ranking. For example, if there were 15 
requirements whose mode was "Attractive," you might use the Self- 
Stated Importance data to rank the Attractive requirements in 
descending order of importance. This would help to further 
discriminate which Attractive requirements the team might want to 
focus on. 


interpretation 


Decisions on what will be included in your product are influenced by 
many factors. As a general guideline, we suggest that you seek to 
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fulfill all Must-Be Requirements, be competitive with market leaders 
on the One-Dimensional Requirements, and include some 
differentiating Attractive elements. 


In general, the return you can expect from fulfilling a requirement (in 
terms of customer satisfaction) should guide the effort you invest to 
fulfill it. Improving performance on a Must-Be Customer Requirement 
which is already at a satisfactory level is not productive when 
compared to improving performance on a One-Dimensional or 

Attractive Requirement. Classifying Customer Requirements into 
Kano's dimensions can help you to focus on the vital few where the most 
leverage exists. 


Additionally, you should use demographic data in conjunction with 
questionnaire analysis to identify potentially differentiating market 
segment characteristics. 


Continuous/Graphical Analysis Approach 


The Statistics Subcommittee of the CQM Research Committee 
considered more sophisticated methods of questionnaire analysis. 
These ideas are presented in Appendix D. 


Tips 
e The time you are taking to hear your customers’ views contributes to 


the company's professional image. The format of the 
questionnaires should further enhance that image. 


e Listen carefully and without bias to what your internal test 
customers say. If they find something confusing, it is quite likely 
that others will as well. Revise questions or add instructions as 
necessary. 


e Establishing the method by which the data will be analyzed 
before disseminating the questionnaires will save time. Will you 
use manual or automated input and analysis tools? Knowing this 
will enable you to marshal the necessary resources while you are 
waiting for replies. 


e When two Kano codes are tied in the scoring for a given question, 
consider: 


1. Following up with customers for additional insight 
2. Looking for market segmentation differences 


3. Selecting the classification that would have the greatest 
impact on the product (use the following ordering: Must-be 
>One-dimensional >Attractive >Indifferent) 


e Itis helpful to get some advice from someone in your firm who has 
experience with customer surveys. 


e A small gift to those who complete the questionnaire may improve 
response rate and generate customer satisfaction! 
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Completion Checklist 


At the end of this step, you should have: 
¢ Compiled and tested questionnaires 
e A mailing list of respondents 


e An analysis template. 
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STEP 8: Generate Metrics for 
Requirements 


This step will identify the metrics which will be used to assess each 
Customer Requirement. You will use these metrics to objectively 
evaluate alternative design solutions in the Concept Selection Stage of 
Concept Engineering. Metrics can also be used to benchmark 
competitive products. Additionally, the process of generating metrics 
increases your understanding of requirements by requiring you to work 
down the ladder of abstraction. 


The team will brainstorm possible metrics for each requirement and 
then select the smallest set of metrics (usually one or two) which cover 
the specific requirement. Some requirements are relatively straight- 
forward (e.g., "the line in the basket is tangle free when cast "). 
Developing a measurement unit to assess these requirements is fairly 
uncomplicated. 


However, some requirements are more ambiguous (e.g., "the basket is 
comfortable"). Developing measurement units for these requirements 
can be difficult, in that any one measurement unit will measure only 
part of the requirement and in addition will also assess some other 
dimension. In these instances multiple metrics may be requried to assess 
one requirement. 


At the end of this step, a tree diagram will be used to organize the set 
of metrics and identify any missing ones. 


Method 


Brainstorm Customer Requirement Metrics 


Working with one Customer Requirement at a time, use traditional 
brainstorming techniques to generate a list of possible metrics which 
could be used to assess this requirement. Try to make this list 
collectively exhaustive; attempt to get all ideas for measuring each 
requirement surfaced for consideration. Develop a brief working 
definition for each possible metric. This definition need only be 
detailed enough to ensure that members of the team have consistent 
interpretations of the metrics’ intent. 


The team can split into small groups or pairs and divide up the 
Customer Requirements to save time. If the team is going to split up, we 
recommend working on one or two Customer Requirements as a full group 
first so that everyone gets a sense of how this is done, and then dividing 
the remaining requirements among teams of two or three members. 
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Evaiuate Vaiidity 


Determine how valid each metric is; validity is the degree to which 
each metric actually measures the requirement, i.e., directly measures 
the requirement rather than measuring something else. Informal 
validity assessments can be made through group consensus rating of 
each potential metric on a high, medium, or low scale. With informal 
assessment, a simple thumbs up for high, thumbs across for medium and 
thumbs down for low vote has worked well. With this approach, select 
the score which receives the most votes. In the event of a tie, select the 
lower value, i.e., select low if there is a tie between medium and low. 


Alternatively, the team could defer to the opinion of a recognized 
expert. Formal validity assessments also can be conducted using 
statistical procedures such as correlation analysis or signal-to- noise 
ratios. 


Evaiuate’ Feasibility 


Feasibility is how easy or difficult it would be to use the metric; to 
actually collect and interpret the data. 


Informally assess the feasibility of using each metric using a high, 
medium and low scale. 


As a guideline in evaluating feasibility, consider briefly describing 
how the data would be collected ( local check sheets, existing reports, 
etc.), and how the data would be analyzed,{ e.g., personal computer 
spreadsheet analysis or mainframe application development). You 
might use this technique for every metric, or when the team has 
difficulty assessing feasibility or disagrees on the feasibility of a 
metric. 


Assess Ambiguity 


In order to select the smallest set of metrics, you need to assess the 
ambiguity of each Customer Requirement. Requirements that are 
ambiguous will probably need more than one metric to assess them well. 
Give each Customer Requirement a rating on an ambiguity scale such as 
the following: 


Everyone Possibilities Everyone 

on team A difference for multiple on team 

interprets in interpretation interpretations agrees on 

differently exists on team exist on team meaning 
1 2 3 4. 3 6 7 
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Select Vital Few Metrics 


For those Requirements with ambiguity ratings in the 5-7 range, select 
the best metric to carry forward to the next step. If you have a highly 
valid, highly feasible metric, then the choice is obvious. If not, use the 
ranking scale below. 


For those Requirements with an ambiguity rating of less than 5 , you'll 
probably need to select more than one metric. Two or three will 
probably be enough to cover the requirement. Select the highest 
ranking metrics which seem to cover the different aspects of the 
requirement. 


vaisy __ ODIOOOOA\A\A 

rastity  DIOCOOA|ACO!A 

rank [2s fa fs fo |7 [8 9 
(*)= Strong C) = Medium A = Weak 

Repeat for each Customer Requirement 

Repeat the process of generating metrics, assessing validity, 


feasibility and ambiguity and selecting the smallest set for each of the 
CRs. 










Stripping Basket Example 


C.R: Line is cast without tangles. 


Validit 


a Count the number of perpendicular loops before cast. 








Count the number of overlapping loops. 


Calculate the percentage of fouled casts 


Calculate the percentage of fouled drops. 


Ambiguity assessment: 6 (everyone basically agrees on interpretation; 
it is straightforward). One metric is selected to carry forward to the 
tree diagram: Calculate the percentage of fouled drops. 
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Construct a Tree Diagram of the selected Metrics 


A Tree Diagram is an analytical tool used for assuring a complete set of 
answers to a “How to” question. The Tree Diagram is often used for 
solution generation, as in “How do we successfully complete Project X”. 
In this step of Concept Engineering, we use the Tree Diagram to answer 
the question, “How do we measure the key Customer Requirements for 
Product X?”. This tool systematically builds a hierarchy of metrics and 
their purposes, and by doing so a team can then step down this 

hierarchy and search for missing or better metrics or groups of metrics. 
An example from the stnipping basket is on page 3.18. 


1. Prepare a Tree Diagram (CQM manual 4P) using the theme: “How 
are the customer's requirements measured?” Use the metrics carried 
forward from earlier work as the first-level labels for the tree. 
Leave much room on the paper for adding new metrics or groups of 
metrics in later steps. 


2. Group each metric by common purpose. Ask, "What's the purpose of 
collecting this data?" 


3. Write a title for each group which completes the sentence, “the 
purpose of using these metrics is to measure 






The purpose of using 

these metrics is to measure 
(fill in the sentence to create 
the title). 


4. Continue the process of grouping by similar purposes and writing 
titles until there are five or fewer groups. 


Top-down checking 


This stage of the Tree Diagram method is quite important, and too often 
teams rush through it and don’t gain the benefit. It is a structured 
brainstorming approach to generating better metrics. Take your time 
and work at generating missing or better metrics. 


1. Start at the top of the tree and work down one level at a time, 
asking, "what, if anything is missing?” For example, if after 
grouping was completed there were three high-level groups of 
metrics, check if these groups assess the full set of customer 
requirements or if something is missing. If you identify a missing 
group, write a title for that group at the proper level of abstraction 
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and then work down the tree, creating titles and metrics as 
appropriate. 


Step down to the next level on the tree, continuing to search for 
omissions. 


At the first title level, continue top-down checking, improving 
existing metrics or adding new ones. 


Evaiuating the metrics 


Once the top-down checking is complete, it is time to evaluate the full 
set of metrics in order to identify the strongest ones. 


il 


Draw an evaluation table on the bottom of the tree which consists 
of one row each for effectiveness, feasibility, and rank, and one 
column for each metric. 


Assess effectiveness of each metric. Ask, “how effective is this 
metric toward achieving the purpose of the title above?” You will 
most likely use high, medium, and low consensus ratings. 


Assess feasibility of each metric. Either use the feasibility rating 
you gave the metric in the brainstorming step, or if the metric is a 
new one which emerged from the top-down checking, then assess 
the feasibility the same way you did earlier. 


Rank the metrics using the same table used to select metrics prior to 
the tree diagram. 


Tips 


It is highly likely that the most effective metrics are not measures 
which are collected and analyzed in the current information 
system. Therefore, don't limit brainstorming to only measures 
currently collected. 


If it is necessary to find stronger metrics, return to the tree diagram 
and look for metrics which are highly effective, but medium or low 
in feasibility. Think about ways to make these metrics more 
feasible. 


This may be a good time to bring others into the team who have 
more knowledge about metrics commonly used to assess customer 
requirements. Balance this against the time necessary to get others 
familiar with the requirements and the process. 


When in doubt about a particular high/medium/low assessment of 
validity or feasibility, err on the low side so that the strongest 
metrics are easier to identify. 


Completion checklist 


At the end of this step, you should have: 


Worksheets that document metrics development 
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e A tree diagram of metrics that covers the full set of Customer 
Requirements. 


Stripping Basket Tree Diagram Example 


Check if basket prevents 
tangles/snags from occurring. 


Check if basket Measure how line 
prevents line covers the basket 
movement in bottom 

bottom of 

Count Measure Measure Measure Measure Count 

the # the height how much ___ outer edge the the 
of loops of loops line falls droop in distribut- number 
over the from out of four wear ion of of loops 

top of bottom of basket positions. loops in in each 

the the when basket section. 

cones/ basket. placed in bottom. 

Stales. it. 






Effectiveness 


Feasibility 
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STEP 9: Integrate Understanding 
Abouf Customer Requirements 


This step captures the knowledge gained to date in the development 
process about the customer's requirements and their metrics. It displays 
the information in a format which allows for clear, concise 
communication to everyone concerned. It is an important reference point 
for downstream development activities. 


This step ends Stage 3 of Concept Engineering and in a sense finishes the 


work in the "requirements space.” The next stage, Concept Generation, 
begins work in the "solution space.” 


Method 


Prepare the Quallty Chart 


1. Convert the Customer Requirement KJ into a tree structure and 
rewrite all of the labels onto 2”x3” post-its. 


ist level 2nd level 3rd level 


Line is stationary 
in basket 


Line placed in the basket 
Stays there until cast 


moves only 
when desired 


Line in the basket 
is tangle free 


Basket naturally gathers 
line in the bottom 


basket easily 
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When required the 
line comes out of the 


Line is cast 
without drag 





2. Place the CR Tree on the left vertical axis of the matrix. 


3. Place the entire Metrics Tree Diagram on the top horizontal axis of 
the matrix. 
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Leave room for two columns just to the right of the requirements for 
the results of the importance and Kano analysis. 


Add one row at the bottom starting, it just to the right of the two 
columns from Step 3, for noting the operational definition 
identification number. 


Draw a grid using the lowest level labels on each axis to determine 
the width of the rows and columns. 


Exampie: Quality Chart 














et elt | a FL 
5 SIS | | Metric to Customer's a 
EEUNSIEL | | | Requirement ap 
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% 3 EST | [| relationship matrix | | | 
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Pi OPDEF Plan Number Lae 


Assess the strength of Requirement to Metric relationship 


Ls 


Start with metrics which were ranked "1" from the evaluation of 
the tree diagram. 


One metric at a time, assess the correlation to each requirement. 
Ask, "How well is requirement measured by metic?" 





Rate each metric as to how well it assesses each requirement on a 
high, medium, or low scale , or blank for no correlation. 
Alternatively, the team could defer to the opinion of an expert. 


Continue with each metric which was rated a "1". Check for 
requirements which don't have a strong correlation to a metric 
(there will most likely be some). If there are requirements without 
a strong metric, go onto metrics rated a "2", and assess each of these 
against the requirements. Check again for requirements which are 
not covered. 
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Continue correlation assessment until all requirements have at least 
one strong metric or until you've assessed all metrics. 


Formal assessments can be conducted using statistical procedures for 
assessing correlation or signal-to-noise ratios. This type of 
assessment would be time-consuming and probably best used in 
situations where the team feels their insight is cloudy, or the team 
feels the need to calibrate its intuition. 


Evaiuate the Matrix 


Select the best set of metrics by following these guidelines: 


If there are any rows for requirements without a strong relationship 
to at least one metric, identify this as an area that needs further 
metrics generation. 


If there are any rows for ambiguous requirements without strong 
relationships to at least two variables, identify this as an area 
that needs further metrics analysis or generation. 


If there are any requirements covered by an excessive number of 
metrics, investigate the potential to eliminate metrics. 


If a metric is correlated with many requirements (i.e., 4 or more), 
this is an indication that the metric is highly abstract and it may 
be difficult to interpret. You'll need to break the metric into 
elements which relate to different requirements. You can use the 
operational definitions which are described next to further define 
the metric so that it directly measures a requirement. 


Deveiop Operational Definitions for Seiected Metrics 


Operational Definitions provide detailed plans for how requirements 
will be assessed. Create an operational definition for each selected 
metric, using the following outline: 


Define the unit of measure 

Define where the data will be collected 
Define when the data will be collected 
Define what specifically will be observed 
Define how the data will be collected 








Define how the data will be displayed 


Define who is responsible for what. 





Finally, test the Operational Definitions by trying to use them. It is 
highly likely that they will require revision after initial trial. 
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Example of an Operational Definition 


OPERATIONAL DEFINITION FOR DROP TIME 


Unit of measure: seconds 
Location: landing #6 on the building E-52 fire escape 


Wrap the first 30' of line (tapered end) around a tube sock and secure 
with two windings of duct tape. 

Strip off enough line (_55") from the reel for the sock to just reach the 
ground when dropped off the landing. 

Strip off another 10' of line from the reel. 

Randomly determine the order of basket configurations to be tested. 
For each configuration: 


—drop the sock over the side; 

-strip the line into the basket without looking; 

—before dropping the sock, look in the basket and assess: 

—the —~distribution of line in the bottom of the basket; 

—the ~% of line above the top of the cones; and 

—the ~number of perpendicular loops; 
Have the timer drop the sock, starting the stop watch when they let 
go and stopping the time when the sock strikes the bottom; 
Record the time it took for the sock to drop as well as any 
observations regarding line payout, such as tangles, excess line 
payout, or others, on the attached check sheet. 
Repeat the process four times for each configuration. 





Reflect 

This is an important time for the team to stop and reflect. You've come 
a long way since Step 1, Plan for Exploration. Think about what you've 
done and what you've learned. What insight has been gained? What 
surprises did you discover? What additional information do you need? 
What would you do differently next time? 


Tips 
¢ Don’t stretch the correlation analysis. There should be many blank 
cells (indicating no correlation) on the Quality Chart. 


e It may not be possible to cover every requirement with a highly 
correlated metric; do the best you can. If the requirement is one of 
critical importance, additional effort at developing a highly valid 
metric may be necessary. 


¢ Quality Charts can be huge, unwieldy wall charts. One way to 
make the chart more manageable is to rewrite the Customer 
Requirements KJ and Metrics tree diagram labels onto onto 2°x3" 
post-its with the 2" side up against the axis for the correlation 
assessment. It will make the chart easier to work with. 
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e The Quality Chart can be used as the basis for conducting 
competitive Benchmarking 


Completion Checklist 
At the completion of this step you should have: 


e A Quality Chart which includes the Customer Requirements, 
metrics and correlation assessment 


¢ Operational Definitions for selected metrics. 
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stage 4: 





Generating 
Concepts 


This stage marks the transition in the development team’s thinking 
from the “requirement or problem space” to the “solution space.” This is 
the stage of Concept Engineering that many development teams 
describe as the opportunity to "finally have some real fun." 


Many of us have been trained to arrive at solutions as quickly as 
possible; good job performance has been, somewhat erroneously, 
equated with quick fixes. Conversely, the first three stages of Concept 
Engineering teach the virtue of patience as applied to product 
development: accurately define requirements before generating 
solutions. 


In Stage 4, the same disciplined thinking process is applied to 
generating potential solutions. The team will systematically 
decompose the development objectives and then cycle between 
independent and group activities to generate and identify the strongest 
solutions. 


The self-documenting nature of Concept Engineering is maintained 


throughout Stage 4, where ideas and combinations of ideas are 
completely preserved for reflection and later review if necessary. 
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Concept Engineering Stages 







Stage 1: Understanding the 
Customer's Environment 








Stage 2: Converting 
Understanding into Customer 
Requirements 









Stage 3: Operationaily 
Defining Requirements for 
Downstream Development 


Stage 4: 
Generating Concepts 


Stage 5: 
Selecting Concepts 
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Stage 4 Steps 


Step 10: 
Decompose the Probiem 


Step 11: 
Generate Ideas 


Step 12: 
Generate Solutions 


In Stage 4 it is desirable to quickly generate many diverse concepts. In 
Stage 5 you will rapidly focus on the most promising solutions and 
converge on the dominant ones. This process is illustrated in the figure 
below. 


Number of 
Concepts 
Co 
n 
ja? 
5° 
you 


TIME 


Stage 4, Generating Concepts, comprises three general steps, as 
described below. 


Step 10, Decompose the Problem, sets out to reduce the complexity of 
the problem by breaking it down into more easily handled subproblems. 
Many different types of decompositions of the same design problem are 
encouraged, to facilitate better coverage of the potential design space. 
Decomposition also allows for individuals or groups to work in parallel 
to develop solution ideas for different components of the product or 
system. 


During Step 11, Generate Ideas, ideas are rapidly and exhaustively 
brainstormed and improved for each of the subproblems. 
Simultaneously, a search for existing solutions is also conducted in order 
to assure complete coverage of the solution space. Lastly, the most 
promising ideas are picked up by group consensus. 


Step 12, Generate Solutions, begins with an exhaustive enumeration of 
solution combinations (combinations of solutions to each of the 
subproblems) followed by a rapid enhancement of the best solutions. 
The solution enhancement is accomplished primarily by the group's 
collective intuition, perhaps aided by some laboratory testing or 
experiments. The output is a set of plausible concepts (solution 
combinations) which serve as input to Stage 5, Concept Selection. 
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step 10: Decompose Design 
Problems 


The purpose of this step is to divide the complex design problem into 
smaller, but independent, subproblems. Multiple diverse 
decompositions are encouraged as an aid for enhancing coverage of the 
potential solution space. You can deepen and broaden your insight from 
the generation of ideas if the design problem is decomposed in diverse 
ways. 


Development Objective 


The first step in decomposing the design problem is to state the 
Development Objective. The Development Objective defines the 
direction of your design efforts. It is a clear, concise articulation of the 
requirements which focuses concept generation. 


Often it is adequate to use the title or the conclusion from the Customer 
Requirements KJ to serve as the basis of the Development Objective. 
For example, the stripping basket Customer Requirements KJ conclusion 
reads: "It should Allow You to Focus on Fishing by Eliminating Line 
Problems and Discomfort.” A translation of the conclusion to a 
Development Objective is: "The Stripping Basket must allow the 
customer to focus on fishing by eliminating line problems and 
discomfort.” 


Write your development objective on a flip-chart paper and hang it 
conspicuously on a wall to remind the team of this focal point. 


Decomposition 


Decompose the problem into its subcomponents. You can dothis in many 
ways. We recommend trying 3 or 4 decomposition possibilities and 
documenting them on flip-chart paper. Start with your organization's 
traditional development decomposition first, that is, how you would 
normally split up the design task. Examples of possible decompositions 
follow: 


¢ Decompose by technical disciplines (mechanical, electrical,etc.) or 
functional components (e.g., inputs, processors, outputs). 
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e Customer Requirements KJ. Decomposing using customer 
requirements provides a customer-oriented perspective. 
Decomposition by first- and second-level titles is recommended. 


e¢ Metrics Tree Diagram. Decomposition by metrics from the Tree 
Diagram provides a technical perspective. Decomposition by first- 
and second-level titles is recommended. 


e A process flow describes the sequence of actions involved before, 
during, and after the use of the product or service. 


Select a minimum of two decompositions for futher development. One 
will usually be your traditional approach and the other should be more 
customer-oriented. 


Coupling 


Coupling allows you to identify problem areas which are independent 
of each other, in which solutions in one area do not effect solutions in 
another area. When the development team judges that the solution 
ideas of two (or more) decomposed subproblems overlap, those problems 
may be linked, or coupled, together. For example, in the stripping 
basket the line tangling and line drag subproblems are not independent; 
one is connected to the other. These requirements can be coupled 
together and solutions can be generated for both subproblems at the 
same time. 


Examples of the Stripping Basket decomposition style are provided 
below. You will notice in the examples which follow that “coupled” 
subproblems are separated by heavy bold lines from the adjacent 
subproblem. 


The decomposition format lists the subproblem as the column heading 
and space below in which to eventually add the brainstormed ideas for 
solutions(Step 11). Use large format flip-chart paper to document your 
decompositions. You'll use the same sheets for Step 11, Idea 
Generation. 
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The first example of coupling is the Stripping basket Customer 
Requirements KJ diagram. This example shows that the next higher 
level title is included in the table to promote clarity to the team. 


Basket prevents 
line problems 


When required, line 
comes out of the basket 


















The line moves only 
when desired. 


Basket accommodates 
casting, stripping, and 


Stripping basket Metrics Tree Diagram. This example shows that the 
next higher level title is included in the table to promote clarity to the 


team. 
Check if basket prevents 
tangles/snags. 
ies eel am 
heck if basket prevents Check if line falls Check if basket prevents 
line movement in bottom naturally into bottom tangling 


Problem Decomposition Example — Stripping Basket Process Flow. 
a basket | Stripping line into | Casting line from Pen 
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Tips 


Work from the Quality Chart developed in Step 9 when 
developing the customer-oriented decompositions. 


Stick with it. You may have difficulty with decomposition; good 
decoupling will pay off later when it will be easier to combine 
solutions from different subproblems. 


Don't be concemed if your team finds that the decomposition by 
functional breakdown or by process flow doesn't allow for coupling 
of subproblems — couples may not exist! 


Before moving forward to Step 11, post the Problem Statement 
sheet on a conspicuous wail to remind the development team of the 
task at hand. 


In order to leave adequate room for the activities of Step 11, use one 
sheet of paper per subproblem or coupled subproblem. Don't be 
alarmed at the stack of flip-chart sheets your team will generate 
during Step 10! 


Completion Checklist 


A clearly articulated Development Objective 
At least two development decompostitions 


Decomposed, independent, subproblems written on large sheets of 
paper. 


215 


step 11: Generate Ideas 


Step 11 forms the bridge between the requirements space of the previous 
steps and the solutions space. Ideas are brainstormed for solutions to 
the subproblems and are combined with existing concepts found through 
researching literature and talking with experts and customers. This 
process should generate an extensive, nearly exhaustive list of possible 
solutions to consider. 


Researching Old Concepts 


In order to identify existing solutions, consider the following resources: 
e Expert analysis 

e Database searches 

¢ Competitive benchmarking 


¢ Solutions gleaned from your customer interviews. 


Expert analysis may be provided by consultants, universities, R & D 
laboratories and your customers. Database resources, such as patent 
searches, on-line periodicals and trade or industry associations may 
produce valuable ideas. You can gain insight by evaluating the "Best In 
Class" or "Best In Industry" products. 


You may choose to split the team up into individuals or pairs to do this 
research. For example, one pair could travel to the library to conduct a 
patent search, another could identify and interview industry experts, 
and still another could identify potential competitive products for 
subsequent team evaluation. 


The results of your research must be clearly documented and assembled 
as a part of the project documentation. 


Generating New Concepts 


The generation of new ideas is a process of unconstrained thinking 
followed by structured reflection. The unconstrained thinking expands 
the boundaries of ideas for evaluation; the structured reflection focuses 
on picking up the strengths and opportunities contained in each idea. 
Initial ideas are seldom the best; push yourself and the team to create 
extensions and hybrids of these early ideas. 
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Expand feasible design space 


Individually, or in a small group, create an exhaustive list of solution 
ideas focusing on one subproblem at a time. 


For each subproblem identified in Step 10, brainstorm ideas which 
expand the boundaries of ordinary idea generation. For example, for 
each subproblem, offer ideas which test: 


e Market constraints. Propose ideas that will flop in the 
marketplace. 


e Technology constraints. Propose ideas achievable only with 
alchemy. 


e Organizational constraints. Propose ideas likely to get you fired. 


Rapidly generate feasible solution ideas which will cover the 
expanded solution space. 


e During this independent generation of ideas, members should view 
the subproblems from each of the product's stakeholder 
perspectives: end-user, dealer, salesperson, installer, etc. 


¢ Solution ideas do not have to be at a specific level of abstraction; 
any idea can contain strengths which can serve as a springboard to 
an even stronger idea. 


e Record each idea on a 3"x3" Post-It and place on the Subproblem 
decomposition worksheet prepared in Step 10. 


A portion of an "Idea Generation” sheet for the Stripping Basket 
subproblem "Line moves only when desired" is shown on the next page. 


Group collaboration 


Meet as a team and: 


e Logically classify and group the previously generated ideas, 
writing titles foreach group. This is similar to the Net-Touching 
process outlined on page 1-10. 


e Review the independent idea generation results, focusing on 
enhancing the strength of the brainstormed ideas, not identifying 
their weaknesses. For each idea listed, ask "what are the 
strengths associated with this idea?” New ideas are added to the 
list as developed. 


e Have another session (10 minutes) of independent idea generation; 
push team members to identify additional ideas. Ideas are posted 
to the sheet on 3"x3" Post-Its. 


e Review the new ideas with a focus on enhancing their strengths. 
For each idea listed, ask "what are the strengths associated with 
this idea?” New ideas are added to the list as developed. 
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e Quickly assess the feasibility of the posted ideas. 


e Select the most promising feasible ideas for each subproblem for 
further development. 


Several methods for selecting the most promising ideas have been used: 


¢ MPM, after discussing the strengths of each idea, to select the best 
three to five ideas. (Some teams go straight to final round 
selection, giving each member only one vote.) 


e Apply DeBono's PMI process. PMI is an attention directing tool. 
Frst note the positive or Plus aspects of an idea, then note the Minus 
and then note the Interesting points of an idea over a period of 
about 2-3 minutes. A PMI worksheet can be found in Appendix (F). 


e An intuitive or consensus decision process by the team. 


Stripping Basket Idea Generation Examples 


Line moves only when desired 


Ideas which would flop 
¢ flat smooth bottom 
¢ plain flexible bottom 


Ideas achievable only with alchemy 
¢ electrostatically charged line 
® magnetically charged line 


Ideas sure to get me fired 
¢ spinning reel(real fly fishermen don't 
use spinning reels) 


Other ideas 

e astro turf 

¢ monofilament stakes 
¢ monofiliment loops 
¢ one traffic cone 

® nails 

e duct tape 
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Tips 


Start idea generation with the customer-oriented decomposition 
first. Complete your traditional decomposition after all 
alternative views have been explored. 


Suzzane Merrit of Polaroid's Creativity Lab suggests the use of a 
“Mental Excursion" as one method for generating additional ideas. 
Use the Image KJ to take a "trip" into the customer's world while 
concentrating on a subproblem. 


Setting specific time limits for idea generation can help 
concentration. 


Plan on several cycles of individual generation and group 
collaboration. Planning several short sessions per week has proven 
useful for some teams. 


Schedule a day for the team to absorb the results of the research on 
old concepts. The presentations from the research should become 
part of the project documentation. 


The first attempts at generating ideas through brainstorming can be 
carried out as a team activity. After the team gets the hang of 
brainstorming in this fashion, subproblems may be allocated to 
individuals or pairs. 


Personal criticism and competition should be strongly discouraged. 


Do not forget to add the ideas you uncovered in your customer visits 
to the list of ideas for evaluation. 


Completion Checklist 


Subproblems and lists of possible solutions are documented on sheets 
of large chart paper; the most promising ideas are highlighted. 


Notes on research of existing concepts. 
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step 12: Generate Solutions 


This is the final step in Concept Generation and it is characterized by 
creativity and reflection as you identify the strongest solution concepts 
from the many ideas generated. Rapid and exhaustive linkages, 
combining ideas from different columns, are made to create solution 
concepts. The use of the summary table maintains the self-documenting 
nature of Concept Engineering by providing an audit trail of the range of 
ideas and idea combinations generated. The strongest solution concepts 
are carried into Stage 5 for more detailed analysis. 


Create a Summary Table for Each 
Decomposition Style 


1. Collect each of the sheets of chart paper containing subproblems 
and their group-selected solutions. Keep them in their 
decomposition groups. 


2. Within a decomposition group, tape the sheets together side-by- 
side, creating one long document. For example, if you decomposed 
using three methods, you will now generate three summary tables. 


Create Solution Concepts 


1. Considering only one summary table (and therefore one 
decomposition approach) at a time, link together subproblem 
solutions into a total concept. In other words, select items from each 
column and combine them to create a solution concept. 


2. Ideally, create as many alternative combinations as possible, 
thereby providing an opportunity for insight and learning. An 
alternative would be for each team member individually to create 
the solution concept they feel is strongest. 


3. Document solution concepts by recording each combination of 
subproblem solutions you generate on a Concept Description Sheet- 
a description of the solution concept in one page or less . 


select the Strongest Solution Concepts 


Review each Solution Concept individually. Eliminate those that are 
apparently infeasible or can readily be discounted by group consensus. 
However, expert evaluation and laboratory testing may be required in 
order to judge the relative strength of many of the other combinations. 
Therefore, the end point of Step 12 and the start point of Step 13 is 
defined as the need for the development team to use resources beyond its 


own intuition in order to accomplish the convergence on the best solution 
concepts. 


Tips 
¢ You'll need long walls in order to accommodate the summary tables; 


plan to use appropriately large facilities for Step 12. 


e When creating solution concepts don't be too critical; focus on 
generating many solution concepts. 


Completion Checklist 
e Summary tables for each decomposition method. 


e¢ Concept Description sheets for each concept to be carried forward to 
Stage 5, Concept Selection. 


e Concept Description sheets for each eliminated concept (save these 
to maintain the completeness of the project documentation 
package). 
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stage 5: Selecting 
the Concept 


In this final stage of Concept Engineering, a product concept is selected. 
In the previous stage, the development team generated a wide array of 
solutions to collectively address the set of Customer Requirements. In 
this stage, the solutions act as raw matenial to be refined, combined, 
and evaluated in an iterative fashion until a small number of complete, 
superior solutions remain. The best concepts are subjected to a final 
evaluation, combining numerical scoring algorithms with the 
considerable intuition built by the development team during its efforts. 
Selection of the dominant concepts consistent with market requirements, 
company capabilities, and company philosophy will be the final result 
of this step. 


In Step 13, Screen Solutions, the team will think individually and 
together, seek expert help, and experiment in the laboratory in an 
iterative process of combining and improving initial solution concepts 
to develop a small number of superior concepts. In Step 14, Select the 
Solution, the "surviving" complete concepts are evaluated in detail, 
using returned Kano and Importance rating data. Two separate 
numerical algorithms can be used to generate scores for each concept. In 
Step 15, Reflect on the Process, the numerical "scores" of the concepts 
are considered in conjunction with other decision factors, to select the 
dominant concepts. 


Concept Engineering Stages 







Stage 1: Understanding the 
Customer's Environment 











Stage 2: Converting 
Understanding into Customer 
Requirements 









tage 3: Operationally 
Defining Requirements for 
Downstream Development 


Stage 5 Steps 





Stage 4: 
Generating Concepts 


Step 13: 
Screen Solutions 





Stage 5: 
Selecting Concepts 


Step 14: 
Select Solution 





Step 15: 
Reflect 
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Step 13: Screen Solutions 


Prior to this step, the team has developed a large number of informally 
screened solutions which have been documented on Concept Description 
sheets. The task of this step is to screen, combine, and improve this set 

of solutions until the best elements have been fused into a small number 

of complete concepts. The tools at hand for this task are: 


e Creative brainpower 
e Experimental work 


e A Concept Screening Matrix. 

At the completion of this step, the remaining mature concepts will be 
ready for final evaluation, to be handled in the next step, Concept 
Selection. 


Screening Process 


The screening process is best explained with the help of a simple flow 
chart. 


Input: Concept Description Sheets 


Evaluate solutions against 
Requirement Space 
Generate New Set of 
Solutions 







Proceed to Concept Selection 


The initial input is the set of selected Concept Description Sheets from 
Step 12, Solution Generation. Some initial solutions may be incomplete, 
addressing only a fraction of requirement space. As iterations of this 
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process are completed, these solutions will address an increasingly 
larger fraction of requirement space. (The Concept Description Sheets 
must be updated accordingly.) Eventually, there will be only a few 
concepts remaining, but these will be complete solutions addressing all 
of the requirement space. When these completed concepts are 
optimized to the satisfaction of the team, Solution Screening is 
complete, and Solution Selection, Step 14 , may begin. 


Evaluate Solutions against Requirement Space 


The term Requirement Space is useful because it suggests the image of a 
map containing the set of customer requirements, along with their 
quality dimension and relative importance. To improve a set of partial 
solutions, there must be some way to evaluate them against this map. 


Screening Matrix 


One way to perform such an evaluation is by evaluating each solution 
(Concept Description) against the set of Customer Requirements 
generated from the Requirements KJ. The resulting screening matrix 
looks something like: 


coerce eee eee Solutions - - --------------- 
2 
5 
a 
A DIE} F] G1] H] I} J 


ai fata anol da 
cx? lo [fwd of al ofl fo [o 
cas [alo [aol al «la fwd ofa 
cae [a Wwalwawal al a wf ala or 
ex.s Walaa [a ofvalwalwalea [2 
xs Ivalee[ 0} of of if afala fr 
cx7 Wwalwal of =f aff apo [a fa 


The solutions are arrayed along the horizontal axis and the Customer 
Requirements are arrayed along the vertical axis. Each cell contains an 
evaluation of how well a particular solution satisfies a given Customer 
Requirement. 
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Reference Concept 


Because the numerical ratings in the screening matrix are relative, 
there must be some reference to compare against. Select a reference 
product. A natural selection might be your current product or a 
competitor's product that you deem "best in class." You can either 
assign the rating 0 to each facet of your reference selection or you can 
evaluate your reference selection, treating 0 as ‘neither a market 
advantage nor a market disadvantage’. In either case, evaluate your 
solutions in reference to the rating you gave to your reference concept. 


Rating Scale 


The example shows evaluations including -2, -1, 0, 1,2 and N/A. The 
higher ratings (1 and 2) reflect superior performance, 0 reflects some 
chosen reference performance, and -2 and -1 reflect sub-reference 
performance. Some teams find the use of -, 0, + provides sufficient 
information about relative performance. The N/A stands for not 
addressed, which simply means that the solution is not complete and 
does not account for that requirement. As more iterations occur, the 
number of N/As in the screening matrix will decrease, and the final 
concepts at the end of Solution Screening should address all 
requirements. 


Alternative Screening Matrix 


The Solution vs. Customer Requirements screening matrix is not the only 
useful screening matrix. To make judgments about how well customer 
requirements are satisfied by a solution can be difficult because it's 
hard to measure customer requirements directly. 


The set of Metrics generated by the team to measure customer 
requirements can be easier to use to assess the solution concepts. The 
complete set of metrics should cover the same requirement space as the 
Requirement KJ. Therefore, it should be as valid to compare solutions 
against metrics from the Metric Tree Diagram as it is to compare 
solutions against requirements with an alternative screening matrix 
like the following one. 


227, 


Reference 


A IB ;|]C;]D|IE[F]G]H J 


Metric ree ee 
mesic? Lo [vif of af of 42}afo [o_ 
movies [42] 0 [+1 | 0} -1| +2] +1 [ral o | +t 
Mewics |-1_fwafnval +2] af -1 [ot | [nal 
Movie S fwal sit | i] ofwalwralnvg +2] 2 
mewics _fwa}e2 | 0} of of a] 2] +1[42 | 
mewic7 [walneal 0 | va] 1] +2] 1] 0 [ot Wa 


An added advantage with metrics is that you may be able to conduct 
relatively quick and simple experiments to determine solutions’ 
effectiveness. 


You can use either or both screening methods. Keep in mind that these 
tools are decision aids: use the approach most appropriate to your 
situation. 


Compieting the Screening Matrix 


Completing the Screening Matrix may involve experimentation. The 
ratings for some cells may be easy to learn or already known. Others, 
however, may require physical modification of devices, quick and dirty 
prototype mockups, or computer modeling. The idea here is not to jump 
full speed into development, but to do just enough lab work to ascertain 
the feasibility and performance of particular solutions (a large number 
of mini-development cycles). 


Generate New Solution Concepts 


After evaluating solutions against the requirement space, you may 
discover that your solutions do a good job in some areas and not in 
others. Create new solution concepts by combining the strengths of 
different existing solutions. 


After completing your first Screening Matrix, talk about each solution 
in turn with the entire group. Discuss the merits and drawbacks of each 
solution. Talk about possible ways to improve solutions and, wherever 
possible, consider combining positive elements from multiple solutions 


into a single solution. Document the new Solution Concepts and 
complete another Screening Matrix. 


After discussion, private thought, evaluation and experimentation, 
ideas have been refined and combined to create a new set of solutions. If 
the team is satisfied that the concepts are complete and there is little 
to be gained by further iterations, proceed to Step 14, Solution 

Selection. Otherwise, take your updated solution set and go through 
another cycle of solution generation and screening. 


Tips 
e Alternate group discussion with individual time for reflection. 


¢ For early screening iterations, the -2,-1,0,1,2 scale may be more 
resolution than is necessary. If you'd rather distinguish only -, 
neutral, and + for early cycles, feel free to do so. However, for later 
iterations, where you are busy fine-tuning the concept, the added 
resolution will be very helpful. 


e Reference concepts are more difficult to generate for completely new 
products. If there is nothing in existence that does what you want 
to do, try to find a set of different products that between them meet 
your customer requirements. Your reference concept will be a hybrid 
of aspects from several products. 


¢ Don't worry if your questionnaires aren't back before you begin 
solution screening. They are unimportant in the early iterations, 
increasingly helpful in later iterations, and absolutely essential in 
Solution Selection. As your concepts become more complete, it 
becomes more important to consider the Kano dimension and the 
relative impartance weighting of your requirements. These 
distinctions will significantly influence the solution “scores” in the 
next step. 


e Feel free to add new solution ideas as inspiration strikes. If they 
are appropriate, they will survive the improvement cycles to 
follow. If not, no harm done. 


e Some teams have found screening with Metrics to be easier than 
screening with Requirements, because metrics are inherently 
quantitative. 


e Make sure group discussions of solutions are frequent. There are 
several benefits. Updated solutions will converge to completed 
concepts faster if everyone develops an intuitive feeling for the 
direction other team members are heading, and can think about 
solution interaction. With frequent communication, you increase 
the tendency for the entire team to reach intuitive consensus on 
final concepts. 
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¢ You may find it helpful to add organizational requirements to the 
screening matrix. For example, resource availability, time, 
manufacturing capability, technological risk, etc., could be useful 
criteria for screening concepts. 


Completion Checklist 
At the end of this step, complete solutions, which address the entire 


requirement space (customer requirements or metrics), should be 
documented on Concept Description Sheets. 
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Step 14: Select the Solution 


Solution Selection is a detailed numerical analysis and scoring of the 
mature concepts developed during Solution Screening. You will use the 
results of these analyses in selecting a product concepts. The basic 
premise of evaluating solutions against the requirement space is the 
same as in.the previous step. However, this final numerical analysis 
explicitly accounts for the results of Kano data and Self-stated 
Importance ratings. This scoring takes into account that some 
requirements are more important than others. 


As in solution screening, two separate Selection matrices are presented, 
one based on customer requirements, the other based on metrics. One or 
both of these matrices may be used to give the team a final detailed 
numerical evaluation of their concepts. 


231 


Process for Selection 
Here is the process flow for Solution Selection: 


Mature Concepts 


Complete Final 
Screening Matrix 


Determine appropriate 
weights 


Score Solutions 


Proceed to Final Step 





Once again, you may choose to assess your solution concepts against 
Customer Requirements or Metrics or both. The Metric algorithm, as 
before, has the advantage of being directly measurable. However, the 
Requirement algorithm is easily adaptable to Kano data. Each will be 
described and an example shown. 


Requirement-Based Scoring 


This scoring method can consider the importance weights and/or the 
Kano results for the set of Customer Requirements. 


Screening Matrix Without Kano Results 
Begin with the screening matrix, Customer Requirements against 
Solutions, (cell-performance ratings are -2,-1,0,1, and 2 as in the 


previous step), with an extra column for Self-Stated Importance 
Questionnaire data. 
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Self-Stated 
Importance 
Ratings A 


Bae eee 

CR. pu fee [or fo fot 
CR.2 2 fo fu fo for 
CR3 ae Eee 
CR. 3 fa fot [42 Jo 


Solution Scores 3 6 12 9 


Scoring Without Kano 


If we included Importance Ratings and not Kano data, we would 
calculate solution scores as the sum of performance ratings, weighted by 
the corresponding Self-Stated Importance Ratings. Solution A, for 
example, would have a score of (+2)(1) + (0)(2) + (+1)(4) + (-1)(3) = 3. 
Those solutions with the highest scores would dominate. 


Screening Matrix Including Kano results: 


To account for Kano results, we must redefine the cell performance 
ratings. For example, if a given requirement has purely attractive 
quality (A) and it is implemented well, customers are very satisfied; if 
implemented poorly or not at all, customers don't care. Suppose we had 
a solution which scored a -2 (very poor implementation) for this 
requirement. With our existing scoring method, the -2 rating would 
detract from the overall effectiveness score for the concept. However, 
we know for Attractive quality, a poor score has no effect on how a 
product is perceived. Thus, the -2 rating for Attractive quality should 
be changed to a 0 rating. Continuing this line of thinking, the various 
Kano results should be redefined as follows: 


A O M I 
Old = -2-1012 22-1012 -2-1012 -2-1012 
New 00012 -2-1012 -2-1000 00000 


Here is a Screening Matrix with Kano data included: 
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Solution Scores 


In this example, 100 people responded to Kano surveys and the team 
considered the top two Kano scores versus just the mode response. Using 
our new ratings, accounting for Kano Dimensions, a scoring example for 
Solution A and C.R. 1 follows: 


The cell performance rating is +2, 60/100 respondents chose M, and 
40/100 respondents chose O, the redefined performance rating is 
(60/100) (0) + (40/100)(+2) = +0.8. This is then multiplied by the Self- 
Stated Importance Rating. 


An example of a complete solution scoring using this method follows: 


----- Solutjons - - - - - 
Self-Stated 
Kano Importance 
Dimension} Ratings A 


CRI eave 1 ta La fe er 
cr2jono | 2 [o |e |2 [+2 
cr3psooso| 4 [+1 fo fio [a 
cr4|M:coro] 3 [4 [+1 [+2 fo 


Solution Scores 3.0 12.4 14.0 18.4 
Solution A Score = [(60/100)(0) + (40/100)(+2)] x1 
+ [(100/100)(0)] x2 


+ [(50/100+1) + (50/100)(+1)] x4 


+ [(60/100)(-1) + (40/100)(0)] = x3 3.0 


Metric-Based Scoring 


This scoring method numerically assesses solution concepts but excludes 
Kano results. Fill out the metric-based Screening Matrix (cell-scores are 
-2,-1,0,1,and 2 as in the previous step), leaving an extra column for 
Metric Importance weights. 


Weight A B C 


D 
ee ee ee 
meric? | fo de | 2 [2 
Mewic3 | fst J 0 | a fot 
Metric 4 po fet fat 2 fo 


Solution Scores 


Determine Metric Importance Weight 


The questionnaires in Step 7 allow you to determine an importance 
rating for each requirement. In this step, you convert that information 
into Metric importance weights by combining the correlation assessment 
from the Quality Chart (Step 9) with the Self-Stated Importance 
results (Step 7). This is a proxy measure of how strongly each metric 
relates to the performance of the entire product from the point of view 
of the customer. As an example, suppose we had the following 
Correlation Matrix: 


------ Metrics - - ---- 
Self-Stated 
Importance 
Ratings l 2 3 4 


CRI pi fo} | jo 
CR? pe) | efor | 


IS 4 
CRA Seo | 


Metric Weighting 17 6 “. 10 
Correlation Weights: @ = High (3) 


© =Medium (2) 
A =bw (1) 
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For each metric, multiply each correlation by the Importance Rating of 
the corresponding requirement. (The Self-Stated Importance Rating is 
the average from all returned surveys for each customer requirement.) 
The sum of these products is the Metric importance weighting. 


For Metric 1, there is a medium correlation with C.R. 1 (correlation 
factor = 2), and the Self-Stated Importance Rating = 5, a high 
correlation with C.R. 3 (correlation factor = 3) and Self-Stated 
Importance Rating = 4, and a low correlation with C.R. 4 (correlation 
factor =1), and Self-Stated Importance Rating = 1. 


Weighting for Metric 1 = (2)(1) + (3)(4) + (1)(3) = 17 


Repeat the calculation for each metric and record the Metric 
Importance on the screening matrix. 


Metric 
Importance 
Weight A |B 


Cc |D 
Metric 
weve? [6 |o Ju [2 [a 
Metric 3 p46 fa fo fae [a 
Metric 4 (10 f-a_[+i [+2 [oo | 


Solution Scores 
Calculate Solutlon Scores 
We now have all the information we need to score each solution. A 
solution's score will be the sum of its cell-performance ratings 
multiplied by the appropriate Metric Importance Weights. 
In our example above, the score for Solution A is 


(+2)(17) + (0)(6) + (+1)(4) + (-1)(10) = 28. 


Here is the screening matrix complete with scores ... 
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Metric 
Importance 
Weight A |B Cab 


eet ia lalo [a 

weer | 6 fo [un [2 [a 

Metric 3 ee eee 

Metric 4 rome eee | non 
28 


Solution Scores 33 116 33 


Solutions B and D are tied with the highest scores. Their strengths lie 
in different areas, but their total effectiveness is rated the same by this 
scoring algorithm. This represents an opportunity to create improved 
solution concepts through combination. 


Tips 
e Itis not necessary to do both types of screening outlined in this step. 


e The creation of additional solution concepts in this stage should be 
encouraged. 


e Don't blindly follow the scoring steps; think about what makes 
sense based on your team's assessment. .You may find -, 0, + is 
satisfactory 

Completion Checklist 

At the completion of this step, a Solution Selection Matrix should be 


completed for every solution concept which received serious 
consideration. 


step 15: Reflect on the Process 


Your team is at the final step of Concept Engineering, where you will 
make a decision on a product concept that you will present and defend 
when asking for resources to support development. At this step you 
should also reflect on the entire Concept Engineering process. 


Decision Factors 


There are many factors to consider in choosing the product concept: 


237 


Solution Scores 


The team has collectively constructed solutions on paper, evaluated 
their performance vs. customer requirements, and scored them with 
appropniate weighting for Kano and Self-stated Importance Rating 
Data. These scores, developed by one or both scoring methods, should 
indicate the dominant concepts. 


intuitive Convergence 


After the entire team has heard the same "voices of the customer," 
mapped out the requirement space, explored the solution space, and 
studied the Kano and Importance data, it is inevitable and beneficial 
that the development team form an opinion about which solution is the 
best concept. This is another basis for making the decision. 


Company Capabilities 

Manufacturing capabilities, distribution channels, resource constraints, 
and product family considerations are all factors which should be 
considered in the selection decision. 


Company Philosophy 


For a product to be approved for development, it must also be consistent 
with company philosophy. It is possible for solutions which meet 
market needs not to be consistent with company philosophy. Thus, 
company philosophy is an important factor to consider in making a 
final decision. 


Final Concept Decision 


Do not blindly select the concept with the highest score. Reflect on 
what you've learned. Consider factors which will ensure acceptance 
and support within the company and the distribution channel. The 
goal is to get your product into the market. The best product concept, if 
implemented poorly or not at all, will not make money. 


Concept Presentation 

You are ready to get commitment to the development of your product 
concept. Your team should have all the documentation needed to 
persuade senior management to support development of the product 
concept. 


Concept Engineering Process Reflection 


As in other TQM methods, Concept Engineering is not finished until the 
team reflects on its work and thinks about improvements to the process. 
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Reflect on how far the team has come, how much has been learned 
about the customers and their requirements. Think about the process, 
what worked well, what was difficult, what could be improved the 
next time. Document these thoughts and pass them along to others in 
the organization and to the CQM so that other companies can benefit 
through mutual learning. 


Tips 

e If your intuition and scores don't match, follow the data trail 
backwards from solution scoring. Pay close attention to the points 
where your intuition disagrees with the highest scoring solutions. 
This may be a clue to an error or omission along the way. When 
these final differences, if any, are ironed out, the team must commit 
to a single concept that they will support and defend. 


e If there are challenges and objections to your concept, you have a 
complete self-documented audit trail showing how you arrived at 
your final solution, along with a very large and defensible list of 
things not to do. 


e Comments on Concept Engineering can be sent to the COM or to Gary 
Burchill at (617) 258-5586 or Diane Shen at (617) 873-3730. 


Completion Checklist 


At the completion of this step you should have commitment from your 
Development and Management team to proceed to detailed 
specification of the selected concept and notes from the team's 
reflection on the process. 
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Appendix A: Glossary 


Concept Engineering: a process for determining customers’ key 
requirements, creating a measurement plan for assessing compliance 
with the requirements, and developing a strong product or service 
concept which satisfies the requirements. Concept Engineering has 5 
stages: Understanding the Customer's Environment, Converting 
Understanding into Customer Requirements, Operationally Defining 
Customer Requirements, Generating Solutions, and Selecting a Solution. 


Concept Description Sheets: a one-page or less description of a 
combination of ideas which make up a solution concept; developed at 
the end of Stage 4, and used in Stage 5, Selecting the Concept. 


Correlation Matrix: one of the Seven Management and Planning Tools 
used to look at the relationship between two sets of items; the Quality 
Chart in Stage 3 of Concept Engineering is a correlation matrix which 
examines the relationship between Customer Requirements and metrics 
in order to choose the smallest set of metrics which covers the set of 
Customer Requirements. 


Customer Requirements Statements: the outcome of translating the 
voice of the customer into requirements; a descriptive sentence stating a 
customer need, not a solution. 


Customer Requirements Worksheet: a four-column sheet for use in 
translating the Voice of the Customer into Customer Requirement 
Statements in Step 4 of Concept Engineering. 


Customer Voices: the first column on the Customer Requirement 
Worksheet; the actual words of the customer; each customer voice is a 
complete thought. 


Decomposition: the first step in Stage 4, Concept Generation, in which 
the team breaks the problem or objective into components by using the 
Requirements KJ, Metrics Tree, process flow, or other methods; solutions 
will be generated for each segment and then combined to form whole 
product concepts. 


Image: a mental picture of an aspect of the customer's environment 
collected from a customer visit, either from something a customer said 
or from an observation of their environment; the second column in the 
Customer Requirements Worksheet, used to link a customer voice to an 
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aspect of the customer's context to help in the interpretation of the 
voice. 


Image KJ: a KJ of acollection of images, or scenes, about the customer's 
environment; a common mental model of the customer's context in which 
the team's product or service will be used. 


Kano Survey: a customer research tool used to learn more about a set of 
Customer Requirements; developed by Noriaki Kano this questionnaire 
is used to determine requirement dimensions, one-dimensional (more 
functionality leads to more customer satisfaction, less functionality 
leads to less satisfaction), Attractive (if present, leads to satisfaction, 
if not present customer is neutral), and Must-Be (if functional, customer 
is neutral, if not functional customer is greatly dissatisfied). 


Key Items: the third column on the Customer Requirements Worksheet; 
the main thoughts or key ideas from the combination of the Customer 
Voice and the image; a bridge from the fuzzy voice of the customer to 
the clear, concise Customer Requirement. 


Ladder of Abstraction: a semantics concept described by S.I. 
Hayakawa; levels at which we select characteristics to describe an 
object of our experience; lower levels on the ladder contain more 
characteristics of one object, terms on higher levels contain fewer 
characteristics of one object to describe what is common among many 
objects; items lower on the ladder are more factual, items higher on the 
ladder are more conceptual. 


Lead Users: a term used by Eric von Hippel to describe users of a 
product or service who have certain characteristics which set them 
apart and make them good targets for product development teams to 
work with in developing product concepts; among these characteristics 
are prototype experience, a strong need for a solution to the problems 
they are experiencing, and an ability to articulate the problem and 
possible solutions. 


Market-In: a concept introduced by Professor Shiba; work is a means 
not an end, the end is customer satisfaction and everyone has a customer. 


Mental Excursions: a technique for generating solution ideas in which 
you concentrate on the area of focus while reflecting on the Image KJ of 
the customer's environment 


Metric Scoring: a scoring method of numerically assessing solution 
concepts in Stage 5 which combines the correlation assessment from the 
Quality Chart with the Self-Stated Importance ratings. 


MPM (Multi-stage Picking-out Method): one of the methods associated 
with KJ developed by Jiro Kawakita; a tool by which a team can 
systematically reduce a large amount of data to the vital few items. 
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Net-Touching: one of the methods developed by Jiro Kawakita, a tool 
for sorting and classifying language data; organization is done logically 
as compared to intuitively in the KJ method; Unlike KJ, Net-touching 
does not lead to new insight but instead helps to quickly give some 
order to a set of language data. 


Operational Definition of a Customer Requirement: a description of a 
metric including how the data will be collected and displayed. 


Plus - Minus - Interesting: an approach for systematically exploring the 
strengths and weaknesses of a solution idea. 


Quality Chart a correlation matrix used in stage 3 to assess the 
metrics against the customer requirements in order to choose the 
smallest set of metrics which will cover the set of requirements; also 
called the "first house of quality.” 


Reference Concept: used in Stage 5, Step 13, Screen Solutions. In order 
to score each potential solution a reference solution is chosen for 
comparison; the reference solution concept can be the "best in class,” the 
current product, or a competitor's product. 


Requirement Importance rating : a way of incorporating the Self- 
Stated ratings into Step 14, Select the Solution; an average of all 
returned Self-Stated surveys. 


Screening matrix: a correlation matrix used in Stage 5, Selecting the 
Concept; it compares solutions against either requirements or metrics 
and by using a reference concept, scores each of the solutions. 


Self-Stated Importance Questionnaire: a customer survey tool; 
customers give a score ranking the importance of each requirement; can 
be used in conjunction with the Kano Survey. 


Solution Concepts: ideas for product or service components are combined 
to form complete solution ideas, or solution concepts. 


Swim in Shallow Water: developed from Shiba's swimming in the 
fishbowl concept; a term used to describe practice customer visits with 
internal or friendly customers before conducting external visits. 


Voice of the Customer: a verbatim transcript of the actual words of 
the customer collected in a face-to face visit or telephone call. 


WV Model: the systematic problem-solving model developed by Jiro 
Kawakita and Professor Shiba, which describes problem-solving as a 
series of steps alternating between the level of thought and the level of 
experience; contains the 7 step reactive problem solving method. 
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Appendix B 


Introduction: 


Stage 1: 
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Appendix C: 
Additional Hints 
on Administering 
Surveys 


Response Rate 


Survey response rates can vary all over the board. For busy 
professionals such as doctors, response rates to mass mailings can be as 
low as one percent. Take heart, though. Generally, response rates are 
much higher among customers (and particularly those you have 
visited). 


You will probably see lower response rates among former customers and 
prospects. Market research firms often maintain statistics on response 
rates sorted by profession. Such information may be useful in planning 
the size of the mailing you need to meet your target response. 


Experience suggests that 95% of those who will respond at all will do so 
in three to four weeks. If you don't have your target response by then, 
you should either follow up or do a supplementary mailing. 


Tips to Improve response rate 

e First Class postage 

e Personalized letter 

e Incentives 

e Post card warning prior to sending questionnaire 


e Post card follow-ups 
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e Professional appearance to survey. This could include any or all of 
the following: 


20 Ib. stock, smooth, non-slick paper, matching envelopes, colors 
(white, off-white, very light gray, beige, olive green ,or 
cream), black ink, same font throughout, boxes to highlight 
sections, hand signature in blue ink, standard size 81/2" x 11", 
flat 8 1/2" x 11" envelope to deliver questionnaire (more 
professional, attracts attention), clean and simple layout, full 
use of white space, use of graphics if warranted, booklet form, 
shading, page numbers in upper nght corner, "Please continue on 
page x" at bottom of page if warranted, note at end thanking 
respondent, title printed at top of each page, color code by 
target group if necessary. 


suggestions for the Cover Letter 


The cover letter should address the following questions: 
e What this is about? 

e Who wants to know? 

e Why do they want to know? 

e Why was I picked? 

e How important is this? 

e Will this be difficult? 

e How long will this take? 

e Will it cost me anything? 

¢ How will this be used? 

e Will I be identified? 

e What's in it for me (benefits, incentive, token of appreciation)? 
e When should I do it (deadline)? 





Example Letter of Introduction 


Dear {company} customer, 


At {company} we recognize that your input must play a vital role if we are to develop products 
that meet your needs. We are writing, therefore, to solicit your views regarding {product} 
features and capabilities with the intention of using your input to enhance {product} or to 
develop new products to better meet your needs. 


The enclosed questionnaire is the result of an effort by the {product} development team to apply 
Total Quality Management methodologies to investigate user requirements in the area of 
{product category}. As part of the method for determining user requirements, we visited a 
representative sample of {product} customers and non-customers, and we recorded their 
comments regarding the types of tasks they are faced with, the problems they encounter and so 
on. We have analyzed these data and developed some conclusions about {product category} 
features and capabilities that users like yourself require, and we would like to check the 
validity of our conclusions by having you answer some questions for us. 


Although some of the questions we ask may appear redundant, they were all carefully chosen 
to gather the information we feel we need to guide us in designing and developing quality 
(product category} that meets your needs. The survey should take about thirty minutes to fill 
out. Please return the completed questionnaire in the enclosed, self-addressed envelope by xxx. 


Sincerely, 


XXXXX 
{Product} Development Manager 


P.S.: To thank you for your time, a {product} T-shirt will be sent to each respondent who 


completes and returns the enclosed questionnaire. The questionnaire includes a space for you to 
indicate your preferred size. 
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Appendix D: Kano 


Understanding Kano Theory 


Kano analysis helps us understand the relationship between the 
fulfillment (or non-fulfillment) of a requirement and the satisfaction or 
dissatisfaction experienced by the customer. Working with social 
science theories on satisfaction developed by Frederick Herzberg, Kano 
discovered that the relationship between fulfillment of a need and the 
satisfaction or dissatisfaction experienced is not necessarily linear 
(although this was the prevailing assumption at the time). He found 
that he could sort requirements into distinct classes, and that each class 
would exhibit a different relationship with respect to satisfaction. 


Kano’s theory sorts customer requirements into five categories 
(described below). A sixth category, "Questionable Result,” indicates a 
potential problem with the questionnaire. A brief definition for each 
of Kano's dimensions is listed below: 


A Attractive Quality Elements: Customer Requirements which create 
satisfaction when fulfilled, but are accepted as is even when not 


fulfilled. In other words, the customer is greatly satisfied when 
this element is present but experiences no dissatisfaction when it is 
not present. In the stripping basket case, the ability to wear the 
basket in different positions was rated Attractive. 


O Qne-Dimensional Quality Elements: Requirements which result in 


rising satisfaction the more they are fulfilled, but lead to 
increasing dissatisfaction when less fulfilled. There is a 
proportional relationship between functionality and satisfaction. 
The water drainage rate is an example of a One dimensional 
element for the stripping basket. 


M Must-Be Quality Elements: Requirements which do not lead to 
satisfaction when fulfilled but cause dissatisfaction when not 


fulfilled. Tangle-free casting was a Must-be element for the 
stripping basket. 


I Indifferent Quality Elements: Requirements which result in 


neither satisfaction nor dissatisfaction regardless of whether they 
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have been fulfilled. Color-fastness was rated as an Indifferent 


requirement for the stripping basket. 


R_ Reverse Quality Elements: Requirements which result in 


dissatisfaction even when fulfilled or in satisfaction even when not 
fulfilled. (This usually indicates a problem in the question.) 


Q Questionable Result: May result from an error in the data- 
gathering process, or from an erroneous assumption about what is a 


functioning (or dysfunctioning) question. 


The relationship between the above three critical categories and the 
satisfaction or dissatisfaction experienced by the customer is expressed 
graphically below. Indifferent elements are represented by the x-axis. 


Satisfaction 


Attractive: 

attachment at 
different body 
positions 










Dysfunctioning —rndifference: -F unctioning 
color fastnes /)—- 
nt 


ar - Must-Be: 
Substanual belt 
tangle free line 
drag free casts 


Dissatisfaction 


-Dimensional: 


water drainage 

corrosion 

resistance. 

neutral color 

line shifts 

natural size loops 

ease of adjustment 

conformity to body 

light weight 

outer edge droop 

interference with 
movement 

sun resistance 

line fallout 

position stability 


It is important to note that the axes in the above diagram are 
asymmetrical, that dissatisfaction is not the opposite of satisfaction. 
When an Attractive requirement goes unfulfilled, the result is not 
dissatisfaction, but simply a lack of satisfaction. In contrast, fulfilling 
a Must-be requirement does not produce satisfaction. It only avoids 





dissatisfaction. Only one class of requirements behaves as if the 
positive and negative axes were continuous: One-dimensional factors. 
These requirements can induce reactions ranging from dissatisfaction, 
through indifference, to satisfaction, depending on how well they are 
fulfilled. 


The Kano question pairs will help us to locate each requirement on the 
above graph. When pieced together, the answers to a 

functioning /dysfunctioning question pair identify which of the five 
categories a given customer need fits into. The process of merging the 
information contained in these response pairs is the purpose of the Kano 
evaluation table described in the section, “Processing Results." As an 
example, though, a response to the functioning question of "I like it that 
way’ generally indicates that the requirement fits somewhere in the 
right side of the above diagram, along the satisfaction axis. The 
response to the dysfunctioning question would pin down the requirement 
as either Attractive or One-dimensional. 


The quality element curves are not necessarily stable over time. The 
curves can shift down and to the right over time as the market becomes 
conditioned to expect certain features. Thus, a feature once considered 
Attractive, such as the remote control for a television set, could move 
over time into the One-dimensional category, and ultimately become a 
Must-be requirement. This phenomenon is termed "quality satisfaction 
decay" by Professor Shoji Shiba. 


In developing and testing the questionnaire, you became familiar with 
the five standard responses to each of the question pairs (ranging from 
"I like it that way” to "I dislike it that way"), and may be curious 
about the order in which they appear. The logic behind the 
arrangement of the responses is the level of pleasure experienced by the 
customer. A scale of pleasure is known as a hedonic scale. 


The question is frequently posed: why is "I like it that way” a stronger 
Statement of pleasure than "It must be that way?” Consider these 
responses in the context of Kano's functioning question. The thought 
behind this ordering is that the first response signifies a type of 
positive satisfaction, while the latter relates to avoidance of 
displeasure. Additional investigation of the hedonic scales is currently 
in progress. 
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Continuous/Graphical Analysis 


There are two powerful advantages to using continuous variables: 


e Acontinuous approach can summarize the data without losing 
resolution. For example, in the Kano evaluation table there are 
nine response pairs which equate to the “Indifferent” 
dimension,and each may have a somewhat different emphasis. 


e A continuous representation deals more comfortably with situations 
where there is no dominant response to a question (e.g., 37% Must- 
Be, 33% Attractive, 30% One-Dimensional) by allowing for 
intermediate points, or hybrids. See the caveat below, however. 


Caveat: The continuous analytical approach described here is best 
applied after inspecting responses for evidence of discrete market 
segments. Lumping together distinctly different perspectives to form an 
average may produce confusing results. Take, for example, a case where 
votes are evenly split between “Attractive,” "Must-Be," and "One- 
Dimensional." One way to check for the existence of distinct segments 
is to runa test of correlation between this response variable and some of 
the demographic data collected with the questionnaire. When 
different segments are identified, we suggest handling the data 
separately. 


To solve the problem of data loss, we can assign each response a 
numenical value, establishing a scale. We propose the following levels 
for the functioning and dysfunctioning responses: 


Functioning Dysfunctioning 
Like=4 Dislike=4 
Must be=2 Live with=2 
Neutral=0 Neutral=0 
Live with=-1 Must be=-1 
Dislike=-2 Like=-2 


Each of the Kano dimensions may now be represented as a coordinate 


pair: 

xX Y 
Reverse -2 -2 
Indifferent 0 0 
One-dim. 4 4 
Must-Be 4 0 
Attractive 0 4 
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Functionality 





These coordinates will serve as reference points for the other data 
(when averages are taken across large data sets, rarely will there be 
“pure” Must-be or Attractive requirements). Answers generally lie 
somewhere in between. With this continuous representation, we can 
retain that information. 


The logic for the asymmetrical scale (beginning from -2, rather than -4) 
is that Must-be and One-dimensional dimensions are stronger responses 
than Reverse, or Questionable. Therefore, our scaling should give less 
weight to such responses to diminish their influence on the average. 
The following map should help to clarify the positioning of the various 
dimensions. You may want to compare it to the Kano evaluation table 
presented earlier. 


Kano Response Map 


Indifferent 
(I) 











Like Must-Be Neutral Live with Dislike 
(-2) (-1) (0) (+2) (+4) 
Dysfunctionality 
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For each Kano question, compute the average of the functional responses 
(these will be mapped on the y-axis) and the average of the 
dysfunctional responses (to be mapped on the x-axis). With a properly 
designed survey, the averages should fall in a range of 0-4, since 
negative values (which indicate questionable or reverse dimensions) 
should be in the minority. 


We can now forma grid (shown opposite), with Xave and Yave ranging 
from 0 to 4 on each axis. Note that each corner of the grid represents a 
prototype, a pure result. For instance, if all respondents to a given 
survey question answered "like" to the functional portion and "dislike" 
to the dysfunctional piece, the coordinates of the average response 
would be (X=+4, Y=+4), indicating a pure One-dimensional requirement. 


Notice that the square is divided into quadrants. When the pairs of 
coordinates representing the average responses to each of the Kano 
questions have been plotted on the grid, the nature of each requirement 
is clearly delineated by the quadrant into which that point falls. For 
instance, a requirement such as number 5, which falls into the upper left 
quadrant, should be viewed as an Attractive element. 


The closer a point falls to one of the four labeled corners (the 
prototypes), the more unanimous the survey respondents must have been 
in their views. Conversely, a point such as number 9, which falls near 
the center of the diagram, is a fuzzier result which indicates 
disagreement among respondents. 


Shading each point according to the average importance vote for that 
requirement is an easy way to integrate the information obtained from 
the Self Stated Importance questionnaire. 


interpretation 


There is no hard-and-fast way to interpret the diagram for 
development prionities. The best approach might vary with the 
number of points falling into each quadrant, with the clustering of the 
points within a quadrant, or with the degree of differentiation of 
importance levels within a quadrant. For instance, in the above 
diagram, there is only one Attractive element. Although it ranks only 
as medium in importance, the team might believe that the product 
needs a differentiating characteristic. 


What makes the most sense is for your group to view the grid (perhaps 
without the points numbered, in order to maintain objectivity about 
which requirements should be pursued), and then agree on a decision 
rule that will work for your data. 
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Average Functionality vs. Average Dysfunctionality 
(Coded by Average Importance, and with Questions Numbered) 


Functionality 





Attractive 


Indifferent 


Dysfunctionality 


Average 
Importance 
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Possible enhancements 


Use the importance votes to weight the Kano responses. Using this 
method means that the more importance a respondent ascribes to a 
given requirement, the greater his or her influence will be in 
classifying the requirement as One-dimensional, Must-Be, etc.. 
Remember to use the weighted version of standard deviation if you 
want to include error bars on this graph. 


Where responses to a particular question have been grouped into 
distinct market segments, plot all the points on one graph, but in 
different colors. If the graph becomes too cluttered, you may have 
to omit the information on variation or importance. To avoid losing 
the importance data, you might vary the color only on the number 
next to each point. 
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Appendix E: Concept 
Engineering 
Process Metrics 


The metrics subcommittee of the CQM Research Committee has been 
investigating product development process metrics. This appendix is 
contained in the Autumn '92 issue of the Center for Quality 
Management Journal. 
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Appendix F: Concept 
Engineering 
Worksheets 





Customer Requirement Worksheet ................. Ee 
Metric Development Worksheet ..................0664. F.3 
Kano Questionnaire Worksheet .................::0008+ F.4 
Self-Stated Importance Worksheet ............... 5 
FAIVAIRVN ONSSIIGOU ceyerierne ss. sse<cs<ntssectecvaes osceceasesieheaendseas. F.6 
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Metric Development Worksheet 


Customer Requirement: 


Ambiguity: 
Possibilities 
Everyone A difference for multiple Everyone 
interprets in interpretation interpretations agrees on 
differently exists exist meaning 





Kano Questionnaire Worksheet 
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. I like it that way. 

. It must be that way. 

. | am neutral. 

. I can live with it that way. 
. I dislike it. 

. I like it that way. 

. It must be that way. 

. | am neutral. 

. I can live with it that way. 
. I dislike it 


. I like it that way. 

. It must be that way. 

. I am neutral. 

. I can live with it that way. 
. I dislike it 

. I like it that way. 

. It must be that way. 

. | am neutral. 

. I can live with it that way. 
. I dislike it. 


. I like it that way. 

. It must be that way. 

. [ am neutral. 

. I can live with it that way. 
. I dislike it. 

. I like it that way. 

. It must be that way. 

. [ am neutral. 

. I can live with it that way. 
. I dislike it. 


. I like it that way. 

. It must be that way. 

. I am neutral. 

. can live with it that way. 
. I dislike it. 


. I like it that way. 

. It must be that way. 

. | am neutral. 

. I can live with it that way. 
. I dislike it. 


. I like it that way. 

. It must be that way. 

. | am neutral. 

. I can live with it that way. 
. I dislike it. 

. I like it that way. 

. It must be that way. 

. I am neutral. 

. I can live with it that way. 
. I dislike it 





Self-Stated Importance Rating Worksheet: 


Not at All Somewhat Very Extremely 


Important Important Important Important Important 
1. How important ts it or would it be tf: 





1 eZ 3 4 3 6 7 8 9 
2. How important ts it or would tt be if: 

1 3 4 5 6 7 Se Y 
3. How important ts tt or would tt be if: 

1 a 3 “ o 6 7 ne) 
4, How important ts it or would it be if: 

1 2 3 a 5 6 F Soe 
5. How important ts it or would it be if: 

1 ie 3 4 5 6 Z ee 
6. How important ts it or would it be if: 

1 y 3 4 5 6 7 8 9 
7. How important 1s it or would it be tf: 

1 2 c A 6 6 7, 8 9 
8. How important ts it or would tt be if: 

1 2 3 Se 5 Cae7 8 9 
9. How important is it or would tt be if: 

1 2 3 4 5 SS ey 8 9 
10. How important is it or would it be if: 

1 Z S 4 ° 6 7 Ba 89 
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Plus - Minus - Interesting Worksheet 


Concept Description: 
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MEETING ANNOUNCEMENT 





The Center for Quality Management 
150 CambridgePark Drive, Cambridge, MA 02140 
Tel#: (617) 873-2152 Fax#: (617) 873-2155 


Please deliver a copy of this fax to each person in your company at the receiving 


fax number listed below. 
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=.an K. Grahan Center fcr Quality Management Hm (508) 369-8714  CQM (617) 873-2155 
Beene Ho retre in: Teracyne, Inc. (ol?) "422-2216 (OR 7) 422-2910 
-.ane Shen Bolc Zeranex and Newman inc (617) 873-3730 (6l7)373-5011 
tary Burcni?: ae (617). 258-5586 (Ol jesse) 579 
“:zesh Parixn Biei-=eeoguremens Cerooracicon 1208) 493-1551 (508) 493-6094 
meeon P. andersss 20 22 ernat tonal (508) 607-4111 (508) 667-9068 
e-¢ Doyle Praxis International Inc. (617) 661-9790 (Gime 97-1072 
7se Kasabula Folaroid Corporation (Clea 2050 (Gia 7 4022 
Bavid Boger Bose Corporation (S08 ee 19-5350 (508) 820-4865 
mugh Loveday Ford Motor Company (313) 322-0886 USisjsc2-4033 
Sartstina Brodie Polaroid Corporation (617) 577-2882 
Sill Fetterman Analoa Devices, Inc. (6174/3937 =2000 
~awn Doughert:-Fitcgerals MIT (Olt jeteoo-5o 77 1 
ean Chan EPA 

“ark Martin MIT fol?) 253-2229 (617s -5 771 
fma<e Timko Analcsa Devices, Inc. (Glos 7 —~ 257 (617) 461-4496 
Base Locker Bolt Seranek and Newman inc. (617) 873-6327 

@_ristopher Mcore Bose Corporation (508) 879-7330 (508) 872-6541 


FROM: Ted Walls PAGE #: 1 DATE: 379/93 


RE: Monthly Meeting 





The CE User's Group minutes and Meeting Announcement are 
attached, with addenda from the meeting. 
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CQM MINUTES 


150 CambridgePark Drive, CE User's Group 
Cambridge, MA 02140 
Tel #: (617)'873-2152 shaxet acto 7 oe Feb 19, 1993 at Bose Corp, Framingham 


In attendance: Ralph Anderson, BTU; Mike Timko, Analog Devices; Elise Locker, 
BBN; Christopher Moon, Bose; Yogesh Parikh, DEC; Jay Thomas, Sippican; Joe 
Kasabula, Polaroid; Gerry Waldron, BTU; Marty Soderlund, BTU; Diane Shen, 
BBN; Gary Burchill, MIT, Mark V. Martin, MIT; Hugh Loveday, Ford; Dawn 
Doherty Fitzgerald, MIT; David Boger, Bose. 


Next Meeting 


Friday, March 19, 1993, from 3 to6 PM, at Polaroid. The CE User's Group meets 
the third Friday of every month. 


Highlights 


1. Mike Timko, (Analog Devices) presented a method for using results of the 
Kano and Self stated Importance Questionaires in Concept Selection. Mike has 
developed an Excel Spreadsheet which uses the "Attractive", "Must Have", One 
Dimensional", and "Indifferent" scores directly with the SSI values to generate the 
“better than", and "worse than" factors. (Eliminates the need to decide the Kano 
category.) 


2. David Boger (Bose) (discussed the use of Concept Engineering Techniques to 
establish "Fitness for Use" during Alpha-Testing of a product. This structured 
method used voice of the customer interviewing of a number of people (12-20) 
who used an engineering sample of a product for a short period. (2 weeks?) 

CE methods were used to "explore" the responses, develop requirements, 
weight and prioritize the opportunities for improvement. 

This Alpha testingwould have been easier if the project had been started 
using CE techniques (SSI and Kano factors available). 

The reaction by the development team to the results was very favorable, 
with appreciation for the clarity of the list. 


3. The announcement for upcoming CE course was reviewed, with minor 
changes suggested. 
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The leaders of the day were were agreed upon as follows: 


Leaders Assistants 

Day 1 May 3 John Petrolini (Teradyne) State Siting Board 

Steps 1,2 Diane Shen (BBN) 

Day 2 May 5 Christina Brodie (Polaroid) State Siting Board 

Steps 3,4 Ralph Anderson (BTU/CQM) 

Payee May 7 Diane Shen (BBN) 
State Siting Board 

Steps 5,6,7 David Boger (Bose) Sippican (J. Thomas) 

Day 4 May 11 Elise Locker (BBN) Sippican (J. Thomas) 

Steps 7,8 Kenny Likis(BBN) sw... Praxis (P.Doyle) 

Day 5 May 13 Joe Kasabula (Polaroid) _......... Praxis (P.Doyle) 

steps9, 10-15 Mark Skilling or Bruce Amazeen 

(Analog) 


note: 1. Yogesh Parikh has offered to fill in where needed 
2. Diane Shen may get substitute for 1 Day. 


Leaders of the Day are requested to have copy ready masters of their material to 
COM by April 14. 


4. Dawn Dougherty Fitzgerald and Mark Martin (MIT Leaders of Manufacturing 
Program) reviewed the feedback from the LOM Concept Engineering Course in 
January. (5 each of 1/2 day session) 

They abstracted the KJ results of the weaknesses--mainly insufficient time 
and inexperience of the instructor with teaching the material. 

Gary Burchill led an effort to capture suggestions for improvement (Labels 
collected by stage of CE) the upcoming course. 
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BUILDING CODES AND SITE MAPS IN MASSACHUSETTS 








| 
| 


Inver City 716 Columbus Avenue (Z1p Code 02120-2193) _. ; COLUMB PTN 8-221-XXXX 
Cambridge: 
* Kenda'l Scuare (Zip Code 02139-1563) oe oN iKS600 PTN 8-221-XXXX 
2 Osborn Street (Zip Code 02139-3591) / a OSB2 . PTN 8-221-XXXX 
21 Oseormn Street 1Zip Coae 92139-3500) ...... 0O§821 PTN 8-221-XXXX 
2b Oshorn Street 1Z19 Coas 02139-3590) |. teay esses 608828 PTN 8-221-XXXX 
38 Henry Street (Zip Coaa 02139-4894) ote, solr PTN 8-221-XX XX 
“19 Winesor Street (Zip Code 02139-3606) . See WR . PTN 8-221-XXXX 
545 Tech Square (Zip Code 02139-3561) wee oa. O45TS PTN 8-221-XXXX 
949 Tech Square (Zip Code 02139-3589) ty ae ee eee §=6§49TS PTN 8-221-XXXX 
963 Tech Square (Zip Code 02139-3586) | . Ree iad: 5857S PTN 8-221-XXXX 
575 Tech Square (Zip Code 02139-3587) eens 97 ols PTN B-221-XXXX 
600 Main Street (Zip Code 02139-3585) wl, 600M PTN 8-221-XXXX 
739 Main Sircot (Zip Code 02139-3584) | _ oe) 6 TO8OM PTN 8-221-XXXX 
rat Main Street (Zi Coce 02139-3583) .. en... ate, FSOM PTN 8-221-XXXX 
734 Meircnal Orive (Zip Code 02139-4687). ......, eee £;' 1. 18) PTN 8-221-XXXX 





~ CENTRAL 
SQUARE 


WE > 
t AVENUE _. 
ne ” 


giOney gtREE! 





To park, you must either get clearance from a guard, or leave 
license at desk in exchange for an access card to the garage. 
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Table 1: Two Dimensional Table of Evaluation 


Dystfunctioning 


2. Must- 4. live 


with 


3. neutral | 


5. dislike 
| 1. like 


2. must-be 
Func- 


: ; | 
tioning | 3. neutral 


4. live with 


5. dislike 





Kano Questionnaire Interpretation 


Satisfaction 
One-Dimensional 
Attractive 
Dysfunctioning | | Functioning 
indifference 
Must-Be 


Dissatisfaction 
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My Kano Questionnaire Interpretation 


Satisfaction Better 


A 


One-Dimensional 
Better 






Attractive 


Dystunctioning Functioning 


Indifferepet 
Must-Be 
Worse 
Worse Dissatisfaction 
A+0O 
Better = _—__-__.~—__xSSl 
A+M+04! 
Worse = eee 


ei Oral 
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Functionality 


Kano Response Data 





Dysfunctionality 


2/7 
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The Use of Concept Engineering Techniques 
To Establish Fitness-for-Use For New 
Products 







CE User's Group Meeting February 19, 1993 





Situation: 
Nearing the completion of new product 
development, Prototypes are field-evaluated 
to test fitness-for-use 


“~—— 


Steo 4 
MPM on Voces 
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step 1: 
Distribute between 12 to 20 devices to 
evaluators 


Purpose. 
* Expose evaluator to oroauct-unaer-test in their home environment 
- Evaluator generates written notes reqgaraing operation 
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LT. — 


Apply results of Griffith & Hauser research 


¢ How to choose evaluators? 
- von Hippie Lead User theory 


¢ Evaluation Time 
- Acceleration of exposure to product-under-test 


- Evaluator’s Notetaking | ae 
- Conscious recail of 80 percent of material lost within 24 hours 
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ATE = 


Step 2: 
Exploration 


Purpose: 
* Collect untiltered information 


CE User's Group Meeting February 19, 1993 


LI s-_- —— 


Simple Discussion Guide 
- What did you like the most? 
- What did you like the least? 
- If you were the designer, what would 
you change? 


Use of Kawakita’s Principles of Collecting 
Language Data 
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Step 3: 
Extract Weakness-Related Voices/Net Touch 


Purpose: 
« Focus on areas not fit for customer use 
> Organize data to streamline downstream processing 
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Step 4: 
MPM on “must-be” issues 


Purpose: i 
- Focus on “must-be” or “showstopper !ssues 
(use Kano's dimensions) 
- Develop a manageable set of issues 
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Step 5: 
Translation 


Purpose 
‘ransiate trom evaluator s language into a form on wnicn the 
Develooment Team can act 
* Create unamoiquous and unrestrictive requirement Statements 









| Step 6: 
Operationaily Define Requirements 





Purpose 
- Objectvety cetine requirements 
- Create system to assess pertormance atternatves 








Opportunities For Improvement 







- Write requirements for each voice. Then MPM requirements. 
not voices. | — | 

* Use Kano survey to weight/prioritize requirements. 

- On CE projects. assess evaluator requirements versus CRs. 


















| 
| * Testing fitness-for-use employs “exploratory” research methods 
- Not “contirmatory’ methogs. 
- In simpiest form, Deveionpment Team is capable of dimensioning 
ISSUES as articulated in “voice” or raw data form. 
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Appendix C: Inductive System Diagrams 


The diagrams in Appendix C come from the Inductive System Diagram 
test conducted in the winter of 1992/1993. The first diagram in a series, i.e. 
Diagram #1.1, is the original diagram with the common variable names 
annotated. The second diagram, i.e. Diagram #1.2, is the revised diagram drawn 


with the common variables. 
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Appendix D: Selected KJ Diagrams 


The KJ diagrams in Appendix D illustrate my approach to selective coding 
analysis for the variable Design Objective Clarity. A complete review of all field 
notes for each team was conducted and references which might be related to 
Design Objective Clarity were selected for consideration. As a result, all KJ data 
labels represent actual observations of, or statements by, product development 
team members. 

The selected KJs represent each of the basic units of the comparative 
analysis. Team 1A came from company 1, used Concept Engineering, and 
exhibited time to MARKET orientation. Team 2A came from company 2, used 
Concept Engineering, and exhibited time to MARKET orientation. Team 2B 
came from company 2, did not use Concept Engineering, and exhibited TIME to 
market orientation. Team 3A came from company 3, used Concept Engineering, 
and exhibited TIME to market orientation. 

The final KJ diagram represents a synthesis of the individual team KJ 
diagrams. The original data labels for this KJ were the first level grouping titles 
from the individual team KJ diagrams. These first level titles were then grouped 
and a second level title was prepared. The data labels shown on the diagram in 


this appendix are actually the second level titles from this analysis. 


Lo, 


What did I observe about design objective clarity from team 1A? 


Concept development is not emphasized 
Product definitions done 


by others are suspect 


Concept development work 
is not recognized as valuable 


Design activities 
can change direction 


This process 
: would have 
: provided a clearer 
2 Vision, a strategic 
: path to the end 
2 itself 


: Stop the number of 

? dead ends. Stop 

? paths with 

: significant right 
tums and left tums 


I just have this 
continuum of fear 
or pressure is that 
knowing while I'm 
doing this there 1s 
no visible output 


Concept 
development is 
not a priority 
activit 


Very difficult to 


2 get sarre sense of 
; urgency from 


: anyone fora 
? product early in 
: its gestation 


The 
development 
stage isn't the 
biggest thing an 
anyone's plate 





Analysis promotes confidence 


Engineers did 
not consider 


function to be 
competent 


Most of the people 
I've worked with 
from marketing | 
would fire 
immediately 


| go to marketing 
and say what are 
the things these 


people are looking 
for. Then! goto 


Marketing and say 
these are the things 
you haven't 
considered 


the marketing 


Product 
development 
specifications 
done by others 
not trusted 


The very fact | 
: had face-to-face 
interviews it 
certainly helps to 
have canfidence 
we aren the 
right path 
How do you know 
the researcher 
wrote what the 
customer said 


: His (marketing 

: rep) worst 

: nightmare is that 
: abunch of 

2 engineers are 


3 selecting the 
‘ product specs 


SSSSSHSHSHSHSSS SESH OSES HOLESS OSS ES ESSERE 


Documented analysis promotes 
management support 


Systematic analysis 
promotes confidence 


Documented Senior Confirmation of Fear is reduced 


audit trails 
reduce 
management 
concerns 


lf the data is 
really good, and 
well-presented, 
the credibility is 
up and the 
attacks are 
down 


By documenting 
a trail perhaps 
you head off 
some of the 
restrictions that 
getimposed on 


you 


COSHH HHHOSHSSHHSEHSSHSHSSHSSSHSSHSSHSOSSS SESH ES SESHEEE 


executives can 
be convinced 


personal 


perspective can be 





when decisions 


with 
documented 
analysis 


I have high 
confidence and 
better ability to 
sell to executive 
comnuttee 
because the 
process is self- 
documenting 


Cancept 
selection process 
provides 
credibility by 
verifying 
intuitive concept 
to someone 


: (PRB) who 

: doesn't 

? understand 

3 what is going on 


with customer 
data increases 
confidence 


Validation of 
what! thought 
was right for the 
market is a net 
gain 


After the 
interviews the 
data is 
somewhat 
different 
although for the 
most part | 
could have put 
down the 
Parameters a 
couple of 
months ago. 
Now Ihave 
higher 
confidence 


supported 
confidently 


{ We will have 


exhausted every 
arguement they 
(PRB) can bring 
before us | 
would no longer 
: feel nervous 


£8 9eoeeeesesseseese 


? If we follow a 
3 process which 
: presents 

2 objective data, 
; you don't have 
: anything to hide 
? from You have 
2 enough data to 
: support your 


cleme B 


Maybe I'm 
spoiled by this 
process because 





Contextual awareness 
improves understanding 


Customer contact 
clarifies requirements 


Requirement 
interpretation 
differences can be 
settled by 
reviewing 
customer context 


They began to 


discuss what a 
label meant and 
went back to the 


and rewrote the 
label 


Disexplaining a 
grouping. 
Others are 
explaining their 
versions. B 
wants to know 


Customer 
exposure 
influences 
designer actions 


{ When you go : 


and get toe-to- 
toe with 
customer it is 
dramatically 
different than 
being in the 
plant 


SSHHHSSHSSHSSHHSHSHSHSHHSESHSHSHOD 


All the stuff 
worked for me 
because | was 
continually 
feeling for the 
customer; every 
input modified 


customer voice 
| 
| 


it has become 
intuitive to me. 
A3F solution bed 
to A4F 


what 1 was 


the context of : 
doing 


. the interview f 


| would dearly 
have loved to 

have one of the 
designers 
involved to get 

an appreciation 

of why you re : 
putting this here : 


orthatthereto : 
getexposure to § 
customer focus 


Confidence in results 


I am willing to 
sign on the 
dotted line as an 
internal spec 


This absolute I get the warm 
belief was and 
unshakable, no feeling 

one could knock 

me off 


PR OS © SEV OWES 6 we PEE COTE SEU BUSES EFS FES SE SE SF OS FO FE ST TT TTT S FU TTS SE 





298 


Individuals determine design outcomes 


Designers must make tradeoffs 


Designers cannot do everything Designers need to 
understand how 


factors interact 


Committed people 
make positive progress 
Designers are 
constrained by 


Designers want 
to focus on the 


Designers want 
to know relative 






Commitment to Passionate 








factors outside _ vital few requirement Designers work design objectives _ participants 
their control requirements importance backwards from promotes design ensure the work 

an image of the realization gets done 
However, can't Try to weed out I have to know finished product 


selection I don't 
have cantrot 
over, and design 
constraints | 
dan't have 
control over 


as many 
possibilities as | 
can early 


We can't do 
everything so 
let's focus an the 
Critical few 


B&B look at the 
remaining labels 
and state: “I'm 
not going to 
vote for 
anymore; I'd 
rather say 21 
than go 24 and 
have some 


bogus 


Discussions are 
used to clarify 


written statements 


: ? which tradeoff 
: : buys more than 
? = It loses 


: | That's where! 

: ? need the input 
: : that tells me 

? : where I need the 


emphasis 









Process participation promotes understanding 


Participation in 
the process 


builds 

















There were 
several different 
discussions 
about what the 
real requirement 
was, eventually 
there was a 
concensus of the 
requirements’ 
meaning 


During Image 
label final round 
MPM selectees 
discussed what 
they saw in the 


labels they select | 





Someone that 
has buyin 


understands the : 


progiany 


understands the ! 


requirements, 


understands the 


how and why, 
and can explain 
to other people 
horizontally or 
vertically 


understanding 






This whole 
process, by 
going through, 
we have 
generated 
separately, we 
all converge on 
the solution 


This process 
allows us all to 
see the data and 


gain buyin 


The outcomes of 
the events we 
have been 
through 
together, 

exclud ing the 
CRK] were 
surprising. 

Now the results 
of the CRK]J are 
not surpnsing to 
me 



































































































] have to get the 
big picture first. 
I have to know 

that it feels nght 
first 





I see the finished 
product and 

then I see myself 
using it and then 
I wnte the specs 





Features are so 
interrelated so | 
could not do it 


ina vacuum 



























Zo9 


{ Post program 
: approved then 


= runin 

: automatic, hard 
? work but people 
> already know 

: what todoand 

? wanttodoit 


: It is safe now. 

: Once it is 

: defined and 

? concensus is 

: built it will stay. 

? I don't have to 

? worry that some 

: mutant child 

? will appear m its 
place 


They have 

0 ; 
They don't want 
to let other 
people down 


I think that 
where there is 
passion there is 
ownership and 
those two 
combmed; when 
they exist in the 
same group of 
people and the 
tear encounters 
problems and 
difficulties they 
don't last 


fight, to do the 
things that 
needed to be 
done 





Individual bias can 
determine the design objectives 


Senior 
managers can 


Personal 
agendas can be 


dictate product pushed dunng 


attributes 


I can see the 
Japanese country ; 
manager comung ; 
up with a 
requirement 

which is : 
completely out of : 
phase : 


The top 


executive threw a : 
product idea out ; 
on the table and 

it was assumed 

as a mandate 


The chairman 
made a decision 
on the spot to 
include x 





design activities 


We've had 
people who lost a 
contract because 
they couldn't do 
a certain 

thing... Takes 
90% of emphasis, 
but the missmg 
item may be only 
1% of customers 


We used to make 
it exacuy the way 
we wnated 
before this 


When pushing 
the marketing 


rep for the 
customer voice 

he stated, “I don't | 
really havea 

voxe for that, | 
made it up 
because | thik 

it's important.” 


March 5, 1993 
Bedford, MA 
Gary Burchill 


What did I observe about design objective clarity from team 2A? 


Understanding is based 
on preceding experiences 


Requirement statements cannot stand alone 


Requirement statements needed clarification 


Placing 
requirement 
statements in 
context of 
original voice can 


clarify intent 
During metric 


During metric > development the 
developmentB i: 3 team 

goes back toa conversations 
worksheet and appear to be 
reads the original : = : directed to 

voice : ; Clarifying 


technical 
We will come characteristics of 
acToOss SO many 


the requirement 

decisian 
pomts—self- J: | am beginning to 
Jocumentation change my frurnd ori 
alowsustogo : the ambiguity 
back to where and : rating we gave this 
what customers : requirement. When 
sad ? we look at the 

: Metric itis not at all 


Metric Product definition activities build on prior efforts 


development 
clarifed the 
intent of some 


requirements 


Ideas are 
connected to the 
results of 
previous work 


Requirements 

were clearly 
Bian linked to 

owing the . 

roots they are customer voices 
going to know 
how we got to 
where we are in 
the product 
definition 


I still find the 
power to be able 
to link one voice 
to one image toa 
key iter toa 
requirement 


When I use 
logical /systemati 
¢ thinking m this 
| think here ts 
what he said and 
here 1s the 
requirement 
which matches 
what he said 


Exposure builds support 





There are ught trade 
offs, severe lirruts to 
what you can do. 
Prioritizing those is 
mainty what you get 
from the customer 


The existence 
of a common 
back ground 
eases conflict 
resolution 


Process 
participation 
builds 
process 
confidence 


Senior 
management 
clearly 
demonstrated 
support for the 
team 





Focus determines effort 


Limited focus concentrates effort They (mkting. & 


eng.) are both 
gomg to know 


The people 

: bitching about the 

? time are not the where the 

: people on the defmitions came 
g ¢ team from and that 
wil] make the 
resolution of 
conflict a lot 
easier 


The GM starts the 
IK] session by 
telling the team hus 
corporate annual 
hoshin dealt with 


$ 
creating value 
f 


Designers want to Discrete bytes of 
focus on the vital information are 


few easier to process 
{ Bis last to select. } Hbewoapte 


i Finally he Sa as ae 
: states,"Nah,I don't :  : 
[a | Saas, 
; w away : 
; without selecting ; the process rather 


: ne : than a huge story 
: e board 


: round of CR MPM, : 


? into individual 
i MT states,"I like ail 3 


: pieces it was 
: of these but I have : important to me to 
to be selective 


? separate every idea : 
? on the paper that 
: allowed us to focus } 


3 reflection oneach ; 
\ ied } 


Clear objectives 
direct designer 
efforts effectively 


: When we got into 

: the room there was 
: built-in credibiilty 
: because the person 
? who is processing 
? the information is 

: witnessing data 

collection 


through VoC 


When it was 
mentioned that 
meal tickets were 
available it brought 
surprise and 


Since we have gone ; 
through this 
process digesting 
requirements and 
looking at 
marketing 


Systematic analysis is thorough 
The team took time Systematic 
analysis reduces 
downstream 
rises 


During the check 


for omission, MT 
states “Ohhh, that 
one is not captured 
anywhere else, put 
it up there 


Understanding 
design objectives 
clarifies what to prontized 


MS states, “Ok, 
everyone happy 
with this?”; MT says, 
“No, we need to 


Unclear design 


objec tives We have covered so 


work on 


A consequence of 
really 
understandmg 
customer 
requirements is 
you know what is 
umportant and 
what is not 
important 

They will know 
what to solve and 
what not to solve 


requirernents tell you 
what 1s good 
enough, where you 
have to put efforts, 
or where you can 


compromuse 


Designers won't 
get otf on a tangent 
spending a week 
or two chasing 
something they 
think 1s neat, they 
will be more 
effective 


reduce 
confidence 


i A consequence of 


: unclear objectives is } 
; you second guess 

: the project during 

: development 


: A consequence of 

: unciear objectives 
: isyouhaveateam ; 
> which questions 


\ Itself } 





thmk about thss one 
for a minute.” 


Some suggestians 
from the team move 
on as the group 
appears to be 
addressed 
elsewhere. MS 


on the group's 


strengths for new 
ideas. Six ideas get 


t 
suggests they reilect 
ee 


much it will be hard : 
to thmk we haven't 
thought about 
something 

Once we decide 

what we are going 

to do there 

shouldn't be any 
surpnses 


| sense a more 
direct path to end 
product—not 
missing things that 
will come back to 
haunt you later 





Designers put themselves first 















Designers typically make 


decisions from their vantage point 





Designers don't Designers typically 
always consider discount inputs from 
the implications non-technical people 
of their decisions 


on others We used to do 
product definition 


Awareness of use 
environment can influence design 






















e get sales’ input 
in design process 
we tend to 
























Actual experiences carry credibili It used to be our by the-seat-of-the- fF wvali 
le DL ty goal was to just pants; Judge info Dt. Poa 
Grae Ereal R ; meet the basic from customer from }. Bee oe 
ones ofre equirements specs without your perspective ‘ : ical insight 
experiences are anchored in worrying about which tends to be 
used to clarify customer what other people {biased Derma lly: oe 
points of experiences have oe Bones aa 
discussion credibilit io aes, re say go ae se 
bea! oh customers an 
MT answers a The second item Asateam during design and come back and tai 
clarification : of credibility is onan fast out later it's the engineers what 
question with a : we have an ered: ae much harder (for the customers want 
reference to an identified group libility that others to deal with) Baar 
the ideas from eng vicers 


existing partand : Of people when 








others were born say he doesn't know 
the problerns or if push comes | toma customer what he's talking 
with it to shove we can se bout J 






go back to need 
In answer a senior 
Manager question 
on performance 
spec the seruor 
engineer answers 
with “Some of the 


An idea comes 


SSCS SSO eSSESES SLES asesesetes 


Traditional development can lose 
sight of customer requirements 





tells a story 
about a 
competition, MS 
tells a story 









Dominant During Traditional 
about a customer ae. 

individuals can development, approach to 

dictate design some capturing 

objectives requirements can customer 






be abandoned comments is 
inefficient 



















Last proyect we 





































worked on we had 
authoritarian eng. All too often you 
Customers view manager who told put blinders on, Led me to believe 







getting ina rut 
attacking one piece 
of the problem and 
letting other aspects 
slide 
We used to set the 


that trying to 
capture what 


customer said with 
a one-liner every 
few minutes and 
looking a week 
later, ridiculously 


us exactly what to 
do whether we 
wanted to or not 
The biggest 
dilemma in the 
past is that a few 


the designed 
part in a broader 
context 




























One level up : 
very strong requirement : =, 
es charters denned (solution) and inefficient 
king at my the product then go off and try Out capture of the 












COT COT COTTE FEE EP TTT STE TE ETFO 


piece but seeing to do it If we 
how my piece fits could not do it, we 
in would make 
decisions that it 
must not be that 


important if we 


whether it was fine 
or not 


(customers) 
answers is 
normally 
sporadic and wee 
lose a lot of these 
answers 


The factors that 
Customers have det 


their own 
ee whether a 
prionites, my part product is 













is only one of : acl cannot do it 
HOW GE 3 different product 
2 Sede = to to juct you 
Demy adou have to adapt ee elie find it 
i t to 
change existing 
desi 


Once you design 
it, it's hard to 
change it, there are 
things you can do 
up front 



















Typically a 
designer starts 
working and it's 
based on his or 
her ideas. Not 
until you doa 
review do you get 
other ideas, it's 
too late to make 
changes then 
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What have I observes about design objective clarity from Team 2B? 


Customers had minimal influence on the design objective 


The marketing function didn’t transfer customer insight effectively 
The design was influenced by factors , 


other than customers 


The marketing rep was reluctant 
to share customer source data 


Marketing reports contained 
less information than 
desired by the engineer 





The marketing 
rep did not 
share his actual 
notes from 
customer visits 
with the 


é marketing 
rep was reluctant 
to give the 
engineer access to 
customer contacts 


The engineer was focused 
on technical considerations 
Direct customer cantact 
provides the engineer 
more information than 


Understanding The engineer 
viewed the 
pertormance 


The engineer 
wanted to 
begin the 
design 
discussion with Taking © cussomers 
you cleo gut images 
thet enake it feed wal 
Having « customer 
ay ‘yooh, that wil be 


confidence 


Ee want © ay thes 
ie wheal were w 
do and wha wrer 
the epecs ore 
what the specs ase 


Customer contact promotes 
engineer's sense of connection 
with actual users 


Another = to jes! 
mor conpected to 
the end applxaton 
you know the name 
of comanne who s 
veing it 





Design objectives were 

influenced by environmental 
influences Early access to an actual design was important to the team 
The designer wanted an actual design 


, The team wanted to start 
prior to discussions with others 


design activities quickly 
An actual 
design was 
the basis for 
discussions 
with other 
designers 


The marketing rep The engineer 
pushed to shorten wanted to start 
the prod uct design 
definition time activities prior 
to having a 
complete 
understanding 


order to talk to 
customers 
about trade-offs M proposes they 
have 75% defired 
al tha next 


You need tobe very (second) moet wg 


quick to get 

soc hang up on tha 
screen, and thet 
becames 3 good 


Wheat I went @ to 
wik w 2 of 3 people 
who have 


experterce briefly to 


R. I'd aay abou s 

can eat 'do n° and week wl get Ue 

I can esy Tan do device for talking to pare a weet to 

x, Dut It will come other dengnam pee pare ee Sister 

ry ] : oe think, but! want to 
You learn around to reepond Wl ee 

Engineer enka to here (company 2) need to hd: cure 

speaks directly © what guts a in Gone 

Customers bes um respotme afd er 

ha ts worred iteorest af haveng & eee hure 

rede oe oon upon ail of the 

goa 


Company employees, 
without the teams 

ex pertence, can 
influence the design 


The taat eoruece of 
confidence B to wlk 
© cumomers after 
you have 8 product 
ddinzion 


You jum have lo 
jump in and 
hang on 


Other designers 
are a primary 
influence on 
the engineer 


Product 
attributes can 
be determined 
by people in 
power who 
lack detailed 
understanding 





The proposed design 
objective was not compelling 


The team 
acknowledged 
the decisions 


could change 


downstream 


The senior 
management group 
was not comumutted to 


the proposed product 


definition 





AUPRB, 1 of} 





You haveto make Resiize you can 


tome dead and alweys go back 
eyree, whetome and change 
wrong ture. if not 
something & wrong 
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At PRB, a lengity 
ducussion sbout 
éubsotugcon whech 
would da tara ne 
the need for 
amoum product 


naruce E ware to 
tagn bpproval 1 of 
J eanwor E coy 
“Las gq we have 
waned enough 
Gere here,” as he 
tagns the approval 











Design objectives were established Data deficiencies affected the decisions 



























Relative priorities are Decisions were made with data deficiencies 


In design you must make choices established during the design 





Decisions were make with incomplete data 







You must make a 
design objective choice 


The team Some product 
recogruzed it attributions 
was making were decided 
decisions with withoul 
cuslomer data 





at you are 


You must state wh 
=O ear confident 


froin to do to ay 

















To get things done, to 
confident. Youcanne say breathe life into it, pou 





know what we ore 


‘1 am thinking about do have to feel confiders dowg. lfwe 
thus’ you have wo easy This erough © my Weare dincover anything 
going to do it about tha 
COMpeuson it s 
likety to be 


weekrams nati @ 


lt is necessary to comumut 
to a product definition 


Alt some point we heve © 
tt, you Neve to focus on emake the decision about 
tt more what type of produce we 
Went to make 







The product definition 
focuses on the key product 
resuirement specs 







(t foa. ft oad ie 
eocnet hing rece 


they'd tie w 











Decisions were made with inappropriate data 


The team guessed at 


the values for some 
decision vanables 






I think about the A product definition is 
vital few end meke just a few key attributes 
them the kil-epecs with eote max/ min 

specs 






Established priorities 
orient trade-off decisions 






bd, Leva mowe on the 
COsnpan bon comps neon of the assum puon 
sheet, Cid you get ane? B: about cna rket aize 
No, I didn’t but of couse furbers ht: Here 
Td be betwee M: Ary clea? are whure the real 
B Toteby e guess amyte i100 wegp come 

naseed of 15 
































definition 
comumnit provides the 
himself only designer 
tothe things = direction with 
he was flexibility 
confident he 


The senior manager 
focused on inappropriate 
data from the decision 

under discussion 












Ai PRE Sw manager goss Se bige goes © board end 
w board and draws greph drove graph MM: The wos't 




















could achieve ee poe ht: This won? help us heip us dectde how cmsy 
decide how many we are we orp powng to sell Se 
important going ro sell E fe this Me. Na, it pull eet the 

aoe but relevent the diecussson spec. We are working om 


comprehensive. doer! know 


it is COmpreherasve 
it dosesrt allow for 
any wede-offe 

The product 
defination gives the 
designer direction 
but | don't wam to 
tell chem how w do 
theis jods 























Informaton availability affects the decision process 












A lack of data undermines 
the decision process 


The availability of data 


Commitments were based increases confidence 


on conservative estimates 











































one a, Ui thie aleence Personal data Data increases 
A lack of data of data, peopie collection the confidence 
T adeliis can create an present promoted data in ideas 
are necessary ines opinions to 
in design opinions differ justify design M4: | went theowgh ofl 
activities objectives we 

















































































and fvo Unset ase 
What @ comes bE fe been eying to apparent X aad C 
down tw 
anking wede-ods keep ary opinion out I was goirg 
of here. but now let's through Ure data 
M: Should we be get ny opireon on the and it dad moa 
record. I cee a lat of match whet arp 
looking for s ¢ or 
piece E Are they C wa have to gat C indead wa insdad Dies wren, 
tmutuely COMmvese Gon with 6 This gives ame ail 
exdvsive? M: Uyeu sr ret sure te cCusaomnaer the cord keane ca 
eee ee! the wortd 
Wall if we cart think wa need this, that Firsthand dara | 
do both, which is Neve asad the In ory qurdd a ceas 


ta what you sy 
Decne ume that i tat 
you thank 


ides hat ecmme coe 
whith hes 0 cheer 


Tove important? quastors and wrote 
down the enw ere, 


Those ere the one | 









A difference in 
leecaemareg toch nical terminology 
definitions existed 





At PRI Alter 4th 
MM: But mnt that the attemp<, EB attempts to 
defination of crenge? « arewers caccurecy 
E: Thara no how | question Four smor 
deme it manages éay “precaaion 
not acturecy.” EB 
accurecy, precasion, the 
wim teng Othen: 
“They ere not” 


March 3, 1993 
MIT 
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What did I learn about design objectives clarity 


from team 3A? 


Individuals did not 
identify themselves 
with team results 

















Individuals had agendas 
unrelated to team efforts 


The team 

















































tried to felt senior 
reduce their ei Sblack managers 
completed. would 

I never Everyone else dictate 

wanted to ba rene solutions 
ut thin no : 

faa ae “How can you without 

cantin't be writing reference to 

deliver, blues without teams effort 

didn't want looking at the 

to get backed {| group?” 

mto a corner 

People right 

now are doing 

anything to 

keep their 

jobs—includin 

g keeping 

their mouths 

shut, doing 

what 

management 

says, whether 

it is nght or 

wrong 






Agreement to move on did not 
mean agreement with results 

































Agreement to proceed does not 
mean agreement with prior 
decision 


















You get a committee W complains 
solution again. It's the least | he cannot 
common think so fast. 
denominator...people don't {| He thinks the 
see anything wrong, but pace 

they are not particularly threatens 
happy with It either “reproducibil 
Discussion of this first iy 


group took 20 minutes. 










People were discussing 

combinations with 

labels outside the Image K] is 

group, and labels were easier because 

shuffled around the no emotional 

board to create attachment, as 
in CRKJ 


hypothetical groupings. 
No one seemed 
particularly satisfied 
with group or title 
when they moved on 













The scope of the project was not 
clearly established at the beginning 


Also there was 
confusion onginally 
whether we were 
talking about Anes 
charts or the entre 
Aries system 


We fluctuated as to 
whether we wanted 


to develop a whole 
product or just a 


package 


>< 









Common understanding 


facilitates 


Common 
experiences 


involvernent, 
are having a 
group of peopie 
who share 
some common 
experience 

lf develop- 
ment and 
torkt hear 
more of the 
same stuff 
from 
customers it 
has to build a 
better 


Interview 
teams had 
skill 


defidenaes 










































interviews did no 


shallow water 
swims 


The core team 
came up witha 
list of 
questions we 
were to ask on 
our visits. My 
personal 
experience was 
that I didn't 
use it 

My first clue 
was when the 
interviewer 
handed mea 
set of his noted 
because he 
wasn't sure the 
note taker was 








oup interactions 


Common 


need to do is get 
us all in a room 
and discuss 
them i 

that don't make 
any sense, we 
would agree to 
let go. 

| kind of saw it 
as we spenta 
lot of time on 
the up front end 
part. | though 
we would have 
less things to 
argue about 
later 


Assignments 
to the team 


were made at 


the last 
minute 


2 of 6 core team 
members who 
did crunch work 
did no 
interviews 


Won't really 
know until the 
15th (KJ) as to 
who is on the 
core tear 


It was almost a 
fluke that I 
was there at 
the training. | 
wasa 
substitute at 
the last 
rmunute. | was 
not clear when 
1 went what 
my role was 
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Process participation builds contextual connections 


Process participation provides more 


documentation 


It is docu- 
mented, but It 
is stil] not the 
Sale; it is not 
like bringing 
one team 
member on 


We have lost 
some people 
who did good 
Visits. We 
have lost what 
they heard, 
what the 
customer did, 
al] we have is 
their notes 


Team members assignments were disruptive 


The team 
experienced 
massive 
turnover 


Every engineer 
assigned to the 
team either 
transferred, quit, 
or was fired 


5 of the 6 people 

icipatng m 
stages 2, 3, and 4 
did not 


































































Sometimes | 
wonder about 
this. When 
you hear 
some thing 
from the 
customer it 
Makes sense. 
If you extract 
it to the 
yellow stickie, 
it’s a problem 








Management did not support the team 





The team perceived a lack of 
management commitment 


To us, gaining a 
better 
understanding of 


the customer's 


problem is 


important, but to 
them (PRB) sowhat wehave 





Management actions 


created 


the attention 


constrained it 


project 


Engineering 
personnel were 
committed to 
other projects 


This didn't get 


Gan to May) in 
particularly of the interview 
engineers process 
because they are . 
tied up on other It is mamly the 
projects ae nied 

of thinking we 
winaverotie | Raden 
ame guidance and 
(VoC visits) we then being told 


are So resource 


means skipping a 


information than process output 


Turnover 
created 
inconsistent 
levels of 
background 
knowledge 


ith turnover we 
will have 
backwards 
movement 
because we will 
have big 
difficulties in 
getting consistent 
thinking on the 
team 
New people will 
not be as 
grounded in the 
customer 
requirement 
background 








At this point it is 
real clear other 
companies have 
much more 
management 
commitment than 









































delays 


Management | 
created a delay 
of several 
months 


















































There was a delay 



















to stop 


Customer's context clarifies requirements 





Abstracted statements can create 
divergent interpretations 
Higher levelsof A statement can 


abstraction be interpreted in 
reduce clarity different ways by 


Requirements anchored in context are clearer 


Customer 
statements 
without 
context 


References to 
customer 


Very often the 
As context resolved | 


discussions 


of original 
intent 


I don't think we 
can get it toa “+° 
or “-" question 
It's too high a 
level of 

abs trac tion 


The higher we go, 
the more we deal 
with semantic 
problems, and we 
redefine the 
things that 
already have 
meaning. You 
can't change the 
Oxford dictionary. 


A CK substantially 
rewntten during 
CRKJ label 


(vagueness) in 
metnc and Kano 
development 


replied that it 
was not, that it 
was information 
oan how to 
analyze data, sort 
of an expert 
system 

A discussion 


different meaning 
to the participants 
He says, “I 
understand the 
words, but! 
don't 

understand 

what they are 
saying” A 
discussion arises 
to further refine 
the title 


entered 
semantics 
problems, not 
the facts. That's 
because the facts 
weren't clear 


decreases 


requirement 
understanding 


When you're 
looking at 
images of VoC, 
and getting 
conclusions in 


requirements is 

the only method 
to clearly define 
a product 


When you do 
Vo, and the 
customer says, 
"1 like x,” if you 
don't 
investigate, you 
don't know 
what the 
message means 








debates 
regarding 
requirement 
understanding / 
meaning 


References to 
expenence in 
customer's 
environment was 
used to clarify 
requirement and 
deveiop metric 
Some confusion 
arose over a 
labeL Some 


participants 
offered their 
interpretation. L 
located and 


reread the 
context this 
settled the matter 
and discussion 


moved to the 
\ next label } 


a 





when it was 
someone 
else's idea, it 
was 

le 
for me to say 
what it means 
When the 
consultant goes 
away, then I am 
the only person 
left. I won't 
have a lot of 
things as to why 
we did this over 
another 


Individuals had an understanding of 


SET © r Ot 


if you can't do 
100%, Le., sell, 
maintain... For 
my real product I 
may want to 
build only 60% 


I think If we come 
up with a project 
that addresses 
90% of that 
(CRKJ) we'll blow 
themaway 





don't know what 
this statement is.” 
We've had some 
that meant more 
to some people 
than others 


"I'm not sure 
what you mean 
by tracking. It's 
unclear to me; is 
It chear to you?” 
“No, this is one | 
wish R was in the 
room because he 
had a lot of 
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The customer's starting point is 
a very concrete way that a thing 
should happen. Ours is always 
a very general method 





Objective clarity focuses efforts 





Clear requirements direct efforts 


Requirement clanty 
promotes focused Defined criteria 


pment effort promote focused 


L” 


have rules, you 
can go m the 
wrong direction 
I'd have a hard 
time eliminating 
more (ideas) 
than one or two 
without clear 
critenia 


If you create a 
new product, 
it's efficient 
because the 
first solution 
you get is 
optimized 


Customer 
empathy builds 
customer 


' By getting mto 


the customer's 
mind, they 
understand the 
product's utility 
Getting oriented to 
customer needs. In 
some cases 
changing biases 
and getting 
intimately familiar 
with customer 
requirements and 
environments 


February 27-28, 1993 
Bedford, MA 
Gary Burchill 


What have I observed about design objective clarity? 


Some design decisions are hard to justify 






















Some design objectives are 


Decisions to move on are : 
not important to customers 


sometimes made just to move on 














Influential Personal 


















Agreements to The analisys is agendas can 
proceed are development not always become design 
sometimes team 1s relevant objectives 





make simply pressured to 


to move on 






There are 
opportunities to 
lose customer 
requirement 
fidelity 

Design objectives 
were established 
with 

data deficiencies 















establishing clear 
design objectives 
dictate product 
design 
objectives 
Concept 

development 

work is not 

given pnonty 

by management 


Concept 
development 
activities are not 
always supported 


by management 





Credible 
engineering 
personnel were 
ot assigned t ° . ‘ : 
ECE Designers make choices during design 





Youcan'tdoall —_ Individuals have a limited 
in one design capacity to focus attention 


One design Designers 

cant decide where 

accomodate they are going 
to concentrate 


Identified 
priorities get 
worked on 


Committed 
individuals meet 


factors outside 
their control 
Trade-offs are 


made during 
development 
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Designers find it 
difficult to 
change existing 
designs 

The designer 
can unilaterally 
decide what to 
emphasize nn 
the design 
Downstream 
development 
activities can 
depart from 
original design 
objectives 


design objectives 
Acknowledged 
priorities keep 
efforts focused 


Clear design 
requirements 
indentify where 
to put 
development 
effort 





Individuals 
can't relate 
equally well to 
all decisians 








Understanding decisions increases confidence 


Context increases understanding 


Background 
information 
clarifies 
written 
statements 


awareness 


builds 


Experiements 
provide more 
information 
than written 
documentatio 


Rhe intent of a 
requirement 
statement can 
be determined 
by customer 
context 


Contextual 


Design 
decisions are 
made in 
relation toa 
larger system 


Disagreements on 
design objectives can exist 


Supportable 
decisions build confidence 


Unsupported 
inferences (opinions) 
create disagreements 


A lack of data can 
create an impasse if 
opinians differ 
Marketing and 


engineering typically 
don't trust each other 


Requirement 
statements can be 
interpreted 
differently 


Distance from context 
increases requirement 
musinterpretation 
opportunities 
Statements can be 
interpreted 
differently 


Supporting 
evidence builds 
management 
comumuttment 


Evidence 
Supporting 
design 
objective 
influences 
management 
committment 


Documented 
analysis 


The ability 
to support 
decisions 
increases 
confidence 


The ability to 
support a 
decision 
increases 
confidence 
Individuals 
wanted to 
make 
conservative 
cormruttments 


increases 
management 
comumnittment 





Decision process 
understanding 
influences 
confidence in 
decision outcomes 


Participation in the 
process can build 
confidence in the 
process results 


Requirements 
linked to 
customers have 
credibility 
Discrete bytes of 
informatian are 
easier to proecss 





March 12, 1993 
MIT 
Gary Burchill 
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